A new algorithm for total variation based image denoising

We propose a new algorithm for the total variation based on image denoising problem. The split Bregman method is used to convert an unconstrained minimization denoising problem to a linear system in the outer iteration. An algebraic multi-grid method is applied to solve the linear system in the inner iteration. Furthermore, Krylov subspace acceleration is adopted to improve convergence in the outer iteration. Numerical experiments demonstrate that this algorithm is efficient even for images with large signal-to-noise ratio.     

Online regularized generalized gradient classification algorithms

This paper considers online classification learning algorithms for regularized classification schemes with generalized gradient. A novel capacity independent approach is presented. It verifies the strong convergence of sizes and yields satisfactory convergence rates for polynomially decaying step sizes. Compared with the gradient schemes, this algorithm needs only less additional assumptions on the loss function and derives a stronger result with respect to the choice of step sizes and the regularization pa...     

A fast algorithm for the total variation model of image denoising

The total variation model of Rudin, Osher, and Fatemi for image denoising is considered to be one of the best denoising models. In the past, its solutions were based on nonlinear partial differential equations and the resulting algorithms were very complicated. In this paper, we propose a fast algorithm for the solution of the total variation model. Our algorithm is very simple and does not involve partial differential equations. We also provide a rigorous proof for the convergence of our algorithm.     

Convergence analysis of gradient descent stochastic algorithms

This paper proves convergence of a sample-path based stochastic gradient-descent algorithm for optimizing expected-value performance measures in discrete event systems. The algorithm uses increasing precision at successive iterations, and it moves against the direction of a generalized gradient of the computed sample performance function. Two convergence results are established: one, for the case where the expected-value function is continuously differentiable; and the other, when that function is nondifferentiable but the sample performance functions are convex. The proofs are based on a version of the uniform law of large numbers which is provable for many discrete event systems where infinitesimal perturbation analysis is known to be strongly consistent.     

A modified spectral conjugate gradient projection algorithm for total variation image restoration

In this study, a modified spectral conjugate gradient projection method is presented to solve total variation image restoration, which is transferred into the nonlinear constrained optimization with the closed constrained set. The global convergence of the proposed scheme is analyzed. In the end, some numerical results illustrate the efficiency of this method.     

A block coordinate gradient descent method for regularized convex separable optimization and covariance selection

We consider a class of unconstrained nonsmooth convex optimization problems, in which the objective function is the sum of a convex smooth function on an open subset of matrices and a separable convex function on a set of matrices. This problem includes the covariance selection problem that can be expressed as an ℓ ₁-penalized maximum likelihood estimation problem. In this paper, we propose a block coordinate gradient descent method (abbreviated as BCGD) for solving this class of nonsmooth separable problems with the coordinate block chosen by a Gauss-Seidel rule. The method is simple, highly parallelizable, and suited for large-scale problems. We establish global convergence and, under a local Lipschizian error bound assumption, linear rate of convergence for this method. For the covariance selection problem, the method can terminate in O(n³/e){O(n^3/\epsilon)} iterations with an e{\epsilon}-optimal solution. We compare the performance of the BCGD method with the first-order methods proposed by Lu (SIAM J Optim 19:1807–1827, 2009; SIAM J Matrix Anal Appl 31:2000–2016, 2010) for solving the covariance selection problem on randomly generated instances. Our numerical experience suggests that the BCGD method can be efficient for large-scale covariance selection problems with constraints.     

Graduated adaptive image denoising: local compromise between total variation and isotropic diffusion

We introduce variants of the variational image denoising method proposed by Blomgren et al. (In: Numerical Analysis 1999 (Dundee), pp. 43–67. Chapman & Hall, Boca Raton, FL, 2000), which interpolates between total-variation denoising and isotropic diffusion denoising. We study how parameter choices affect results and allow tuning between TV denoising and isotropic diffusion for respecting texture on one spatial scale while denoising features assumed to be noise on finer spatial scales. Furthermore, we prove existence and (where appropriate) uniqueness of minimizers. We consider both L ² and L ¹ data fidelity terms.     

The algorithms about fast non-localmeans based image denoising

Image denoising is still a challenge of image processing.Buades et al.proposed a nonlocal means (NL-means) approach.This method had a remarkable denoising results at high expense of computational cost.In this paper,We compared several fast non-local means methods,and proposed a new fast algorithm.Numerical experiments showed that our algorithm considerably reduced the computational cost,and obtained visually pleasant images.     

Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions

Classical stochastic gradient methods are well suited for minimizing expected-value objective functions. However, they do not apply to the minimization of a nonlinear function involving expected values or a composition of two expected-value functions, i.e., the problem \(\min _x \mathbf{E}_v\left[ f_v\big (\mathbf{E}_w [g_w(x)]\big ) \right] .\) In order to solve this stochastic composition problem, we propose a class of stochastic compositional gradient descent (SCGD) algorithms that can be viewed as stochastic versions of quasi-gradient method. SCGD update the solutions based on noisy sample gradients of \(f_v,g_{w}\) and use an auxiliary variable to track the unknown quantity \(\mathbf{E}_w\left[ g_w(x)\right] \). We prove that the SCGD converge almost surely to an optimal solution for convex optimization problems, as long as such a solution exists. The convergence involves the interplay of two iterations with different time scales. For nonsmooth convex problems, the SCGD achieves a convergence rate of \(\mathcal {O}(k^{-1/4})\) in the general case and \(\mathcal {O}(k^{-2/3})\) in the strongly convex case, after taking k samples. For smooth convex problems, the SCGD can be accelerated to converge at a rate of \(\mathcal {O}(k^{-2/7})\) in the general case and \(\mathcal {O}(k^{-4/5})\) in the strongly convex case. For nonconvex problems, we prove that any limit point generated by SCGD is a stationary point, for which we also provide the convergence rate analysis. Indeed, the stochastic setting where one wants to optimize compositions of expected-value functions is very common in practice. The proposed SCGD methods find wide applications in learning, estimation, dynamic programming, etc.     

Proximity algorithms for the L1/TV image denoising model

This paper introduces a proximity operator framework for studying the L1/TV image denoising model which minimizes the sum of a data fidelity term measured in the ?¹-norm and the total-variation regularization term. Both terms in the model are non-differentiable. This causes algorithmic difficulties for its numerical treatment. To overcome the difficulties, we formulate the total-variation as a composition of a convex function (the ?¹-norm or the ?²-norm) and the first order difference operator, and then express the solution of the model in terms of the proximity operator of the composition. By developing a “chain rule” for the proximity operator of the composition, we identify the solution as fixed point of a nonlinear mapping expressed in terms of the proximity operator of the ?¹-norm or the ?²-norm, each of which is explicitly given. This formulation naturally leads to fixed-point algorithms for the numerical treatment of the model. We propose an alternative model by replacing the non-differentiable convex function in the formulation of the total variation with its differentiable Moreau envelope and develop corresponding fixed-point algorithms for solving the new model. When partial information of the underlying image is available, we modify the model by adding an indicator function to the minimization functional and derive its corresponding fixed-point algorithms. Numerical experiments are conducted to test the approximation accuracy and computational efficiency of the proposed algorithms. Also, we provide a comparison of our approach to two state-of-the-art algorithms available in the literature. Numerical results confirm that our algorithms perform favorably, in terms of PSNR-values and CPU-time, in comparison to the two algorithms.     

Feasible descent algorithms for mixed complementarity problems

In this paper we consider a general algorithmic framework for solving nonlinear mixed complementarity problems. The main features of this framework are: (a) it is well-defined for an arbitrary mixed complementarity problem, (b) it generates only feasible iterates, (c) it has a strong global convergence theory, and (d) it is locally fast convergent under standard regularity assumptions. This framework is applied to the PATH solver in order to show viability of the approach. Numerical results for an appropriate modification of the PATH solver indicate that this framework leads to substantial computational improvements. Received April 9, 1998 / Revised version received November 23, 1998?Published online March 16, 1999     

Fast gradient descent method for Mean-CVaR optimization

We propose an iterative gradient descent algorithm for solving scenario-based Mean-CVaR portfolio selection problem. The algorithm is fast and does not require any LP solver. It also has efficiency advantage over the LP approach for large scenario size.     

A unifying approach to isotropic and anisotropic total variation denoising models

Total variation minimisation is a well-established method for digital image restoration. Its implicit preservation of edges permits the derivation of anisotropic models for a qualitative improvement at corners. This paper is a synopsis of anisotropic models with state-of-the-art insights into the numerics of isotropic models. We generalise two representative models from both branches of research. This formulation leads to a general convergent algorithm and a general highly efficient algorithm which apply for both cases. A transfer of the discretisation from the anisotropic model to the isotropic setting results in an improvement of rotational invariance.     

Domain decomposition methods with graph cuts algorithms for total variation minimization

Recently, graph cuts algorithms have been used to solve variational image restoration problems, especially for noise removal and segmentation. Compared to time-marching PDE methods, graph cuts based methods are more efficient and able to obtain the global minimizer. However, for high resolution and large-scale images, the cost of both memory and computational time increases dramatically. In this paper, we combine the domain decomposition method and the graph cuts algorithm for solving the total variation minimizations with L ¹ and L ² fidelity term. Numerous numerical experiments on large-scale data demonstrate the proposed algorithm yield good results in terms of computational time and memory usage.     

A combination model for image denoising

In this paper, we propose an efficient combination model of the second-order ROF model and a simple fourth-order partial differential equation (PDE) for image denoising. The split Bregman method is used to convert the nonlinear combination model into a linear system in the outer iteration, and an algebraic multigrid method is applied to solve the linear system in the inner iteration. Furthermore, Krylov subspace acceleration is adopted to improve convergence in the outer iteration. At the same time, we prove that the model is strictly convex and exists a unique global minimizer. We have also conducted a variety of numerical experiments to analyze the parameter selection criteria and discuss the performance of the fourth-order PDE in the combination model. The results show that our model can reduce blocky effects and our algorithm is efficient and robust to solve the proposed model.