Automatic Smoothing With Wavelets for a Wide Class of Distributions

Wavelet-based denoising techniques are well suited to estimate spatially inhomogeneous signals. Waveshrink (Donoho and Johnstone) assumes independent Gaussian errors and equispaced sampling of the signal. Various articles have relaxed some of these assumptions, but a systematic generalization to distributions such as Poisson, binomial, or Bernoulli is missing. We consider a unifying l₁-penalized likelihood approach to regularize the maximum likelihood estimation by adding an l₁ penalty of the wavelet coefficients. Our approach works for all types of wavelets and for a range of noise distributions. We develop both an algorithm to solve the estimation problem and rules to select the smoothing parameter automatically. In particular, using results from Poisson processes, we give an explicit formula for the universal smoothing parameter to denoise Poisson measurements. Simulations show that the procedure is an improvement over other methods. An astronomy example is given.     

Compression and denoising using <Emphasis Type="Italic">l</Emphasis><Subscript>0</Subscript>-norm

In this paper, we deal with l ₀-norm data fitting and total variation regularization for image compression and denoising. The l ₀-norm data fitting is used for measuring the number of non-zero wavelet coefficients to be employed to represent an image. The regularization term given by the total variation is to recover image edges. Due to intensive numerical computation of using l ₀-norm, it is usually approximated by other functions such as the l ₁-norm in many image processing applications. The main goal of this paper is to develop a fast and effective algorithm to solve the l ₀-norm data fitting and total variation minimization problem. Our idea is to apply an alternating minimization technique to solve this problem, and employ a graph-cuts algorithm to solve the subproblem related to the total variation minimization. Numerical examples in image compression and denoising are given to demonstrate the effectiveness of the proposed algorithm.     

A Parameter Selection Method for Wavelet Shrinkage Denoising

Thresholding estimators in an orthonormal wavelet basis are well established tools for Gaussian noise removal. However, the universal threshold choice, suggested by Donoho and Johnstone, sometimes leads to over-smoothed approximations.For the denoising problem this paper uses the deterministic approach proposed by Chambolle et al., which handles it as a variational problem, whose solution can be formulated in terms of wavelet shrinkage. This allows us to use wavelet shrinkage successfully for more general denoising problems and to propose a new criterion for the choice of the shrinkage parameter, which we call H-curve criterion. It is based on the plot, for different parameter values, of the B ₁ ¹(L ₁)-norm of the computed solution versus the L ₂-norm of the residual, considered in logarithmic scale. Extensive numerical experimentation shows that this new choice of shrinkage parameter yields good results both for Gaussian and other kinds of noise.     

On Smoothing l1 Exact Penalty Function for Constrained Optimization Problems

This article introduces a smoothing technique to the l₁ exact penalty function. An application of the technique yields a twice continuously differentiable penalty function and a smoothed penalty problem. Under some mild conditions, the optimal solution to the smoothed penalty problem becomes an approximate optimal solution to the original constrained optimization problem. Based on the smoothed penalty problem, we propose an algorithm to solve the constrained optimization problem. Every limit point of the sequence generated by the algorithm is an optimal solution. Several numerical examples are presented to illustrate the performance of the proposed algorithm.     

A second-order smooth penalty function algorithm for constrained optimization problems

This paper introduces a second-order differentiability smoothing technique to the classical l ₁ exact penalty function for constrained optimization problems(COP). Error estimations among the optimal objective values of the nonsmooth penalty problem, the smoothed penalty problem and the original optimization problem are obtained. Based on the smoothed problem, an algorithm for solving COP is proposed and some preliminary numerical results indicate that the algorithm is quite promising.     

Minimised Geometric Buchberger Algorithm for Integer Programming

Recently, various algebraic integer programming (IP) solvers have been proposed based on the theory of Gröbner bases. The main difficulty of these solvers is the size of the Gröbner bases generated. In algorithms proposed so far, large Gröbner bases are generated by either introducing additional variables or by considering the generic IP problem IP _A,C . Some improvements have been proposed such as Hosten and Sturmfels' method (GRIN) designed to avoid additional variables and Thomas' truncated Gröbner basis method which computes the reduced Gröbner basis for a specific IP problem IP _A,C (b) (rather than its generalisation IP _A,C ). In this paper we propose a new algebraic algorithm for solving IP problems. The new algorithm, called Minimised Geometric Buchberger Algorithm, combines Hosten and Sturmfels' GRIN and Thomas' truncated Gröbner basis method to compute the fundamental segments of an IP problem IP _A,C directly in its original space and also the truncated Gröbner basis for a specific IP problem IP _A,C (b). We have carried out experiments to compare this algorithm with others such as the geometric Buchberger algorithm, the truncated geometric Buchberger algorithm and the algorithm in GRIN. These experiments show that the new algorithm offers significant performance improvement.     

The Exactness Property of the Vector Exact l1 Penalty Function Method in Nondifferentiable Invex Multiobjective Programming

In this article, the vector exact l₁ penalty function method used for solving nonconvex nondifferentiable multiobjective programming problems is analyzed. In this method, the vector penalized optimization problem with the vector exact l₁ penalty function is defined. Conditions are given guaranteeing the equivalence of the sets of (weak) Pareto optimal solutions of the considered nondifferentiable multiobjective programming problem and of the associated vector penalized optimization problem with the vector exact l₁ penalty function. This equivalence is established for nondifferentiable invex vector optimization problems. Some examples of vector optimization problems are presented to illustrate the results established in the article.     

Smoothed penalty algorithms for optimization of nonlinear models

We introduce an algorithm for solving nonlinear optimization problems with general equality and box constraints. The proposed algorithm is based on smoothing of the exact l ₁-penalty function and solving the resulting problem by any box-constraint optimization method. We introduce a general algorithm and present theoretical results for updating the penalty and smoothing parameter. We apply the algorithm to optimization problems for nonlinear traffic network models and report on numerical results for a variety of network problems and different solvers for the subproblems.     

Linear programming via a quadratic penalty function

We use quadratic penalty functions along with some recent ideas from linearl ₁ estimation to arrive at a new characterization of primal optimal solutions in linear programs. The algorithmic implications of this analysis are studied, and a new, finite penalty algorithm for linear programming is designed. Preliminary computational results are presented.Research supported by grant No. 11-0505 from the Danish Natural Science Research Council SNF.     

A new exact exponential penalty function method and nonconvex mathematical programming

A new exact penalty function method, called the l₁ exact exponential penalty function method, is introduced. In this approach, the so-called the exponential penalized optimization problem with the l₁ exact exponential penalty function is associated with the original optimization problem with both inequality and equality constraints. The l₁ exact exponential penalty function method is used to solve nonconvex mathematical programming problems with r-invex functions (with respect to the same function η). The equivalence between sets of optimal solutions of the original mathematical programming problem and of its associated exponential penalized optimization problem is established under suitable r-invexity assumption. Also lower bounds on the penalty parameter are given, for which above these values, this result is true.     

An inexact algorithm for composite nondifferentiable optimization

We describe an inexact version of Fletcher's QL algorithm with second-order corrections for minimizing composite nonsmooth functions. The method is shown to retain the global and local convergence properties of the original version, if the parameters are chosen appropriately. It is shown how the inexact method can be implemented, for the case in which the function to be minimized is an exact penalty function arising from the standard nonlinear programming problem. The method can also be applied to the problems of nonlinearl ₁ - andl -approximation.This research supported in part by the National Science Foundation under Grant DMS-8619903, and by the Air Force Office of Scientific Research under Grant AFOSR-ISSA-870092.     

Discontinuous piecewise linear optimization

A theoretical framework and a practical algorithm are presented to solve discontinuous piecewise linear optimization problems dealing with functions for which theridges are known. A penalty approach allows one to consider such problems subject to a wide range of constraints involving piecewise linear functions. Although the theory is expounded in detail in the special case of discontinuous piecewiselinear functions, it is straightforwardly extendable, using standard nonlinear programming techniques, tononlinear (discontinuous piecewise differentiable) functions.The descent algorithm which is elaborated uses active-set and projected gradient approaches. It is a generalization of the ideas used by Conn to deal with nonsmoothness in thel ₁ exact penalty function, and it is based on the notion ofdecomposition of a function into a smooth and a nonsmooth part. The constrained case is reduced to the unconstrained minimization of a (piecewise linear)l ₁ exact penalty function. We also discuss how the algorithm is modified when it encounters degenerate points. Preliminary numerical results are presented: the algorithm is applied to discontinuous optimization problems from models in industrial engineering. © 1998 The Mathematical Programming Society, Inc. Published by Elsevier Science B.V.Supported by the Natural Sciences and Engineering Council of Canada and the Centre de Recherches Mathématiques, Université de Montréal.This research was supported in part by the Advanced Research Projects Agency of the Department of Defense and was monitored by the Air Force Office of Scientific Research under Contract No. F49620-91-C-0079. The United States Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright notation hereon.     

The Convergence of a Levenberg–Marquardt Method for Nonlinear Inequalities

In this paper, we consider the least l ₂-norm solution for a possibly inconsistent system of nonlinear inequalities. The objective function of the problem is only first-order continuously differentiable. By introducing a new smoothing function, the problem is approximated by a family of parameterized optimization problems with twice continuously differentiable objective functions. Then a Levenberg–Marquardt algorithm is proposed to solve the parameterized smooth optimization problems. It is proved that the algorithm either terminates finitely at a solution of the original inequality problem or generates an infinite sequence. In the latter case, the infinite sequence converges to a least l ₂-norm solution of the inequality problem. The local quadratic convergence of the algorithm was produced under some conditions.     

The Solution of a Problem of Coifman, Meyer, and?Wickerhauser on Wavelet Packets

Wavelet packets provide an algorithm with many applications in signal processing together with a large class of orthonormal bases of L ²(ℝ), each one corresponding to a different splitting of L ²(ℝ) into a direct sum of its closed subspaces. The definition of wavelet packets is due to the work of Coifman, Meyer, and Wickerhauser, as a generalization of the Walsh system. A question has been posed since then: one asks if a (general) wavelet packet system can be an orthonormal basis for L ²(ℝ) whenever a certain set linked to the system, called the “exceptional set” has zero Lebesgue measure. This answer to this question affects the quality of wavelet packet approximation. In this paper we show that the answer to this question is negative by providing an explicit example. In the proof we make use of the “local trace function” by Dutkay and the generalized shift-invariant system machinery developed by Ron and Shen.     

Fused-MCP With Application to Signal Processing

ABSTRACT

Friedman et al. proposed the fused lasso signal approximator (FLSA) to denoise piecewise constant signals by penalizing the ?₁ differences between adjacent signal points. In this article, we propose a new method, referred to as the fused-MCP, by combining the minimax concave penalty (MCP) with the fusion penalty. The fused-MCP performs better than the FLSA in maintaining the profile of the original signal and preserving the edge structure. We show that, with a high probability, the fused-MCP selects the right change-points and has the oracle property, unlike the FLSA. We further show that the fused-MCP achieves the same l₂ error rate as the FLSA. We develop algorithms to solve fused-MCP problems, either by transforming them into MCP regression problems or by using an adjusted majorization-minimization algorithm. Simulation and experimental results show the effectiveness of our method. Supplementary material for this article is available online.     

An interior-point piecewise linear penalty method for nonlinear programming

We present an interior-point penalty method for nonlinear programming (NLP), where the merit function consists of a piecewise linear penalty function and an ? ₂-penalty function. The piecewise linear penalty function is defined by a set of break points that correspond to pairs of values of the barrier function and the infeasibility measure at a subset of previous iterates and this set is updated at every iteration. The ? ₂-penalty function is a traditional penalty function defined by a single penalty parameter. At every iteration the step direction is computed from a regularized Newton system of the first-order equations of the barrier problem proposed in Chen and Goldfarb (Math Program 108:1?C36, 2006). Iterates are updated using a line search. In particular, a trial point is accepted if it provides a sufficient reduction in either of the penalty functions. We show that the proposed method has the same strong global convergence properties as those established in Chen and Goldfarb (Math Program 108:1?C36, 2006). Moreover, our method enjoys fast local convergence. Specifically, for each fixed small barrier parameter???, iterates in a small neighborhood (roughly within o(??)) of the minimizer of the barrier problem converge Q-quadratically to the minimizer. The overall convergence rate of the iterates to the solution of the nonlinear program is Q-superlinear.     

A study on orthogonal two-direction vector-valued wavelets and two-direction wavelet packets

In this paper, the notion of two-direction vector-valued multiresolution analysis and the two-direction orthogonal vector-valued wavelets are introduced. The definition for two-direction orthogonal vector-valued wavelet packets is proposed. An algorithm for constructing a class of two-direction orthogonal vector-valued compactly supported wavelets corresponding to the two-direction orthogonal vector-valued compactly supported scaling functions is proposed by virtue of matrix theory and time-frequency analysis method. The properties of the two-direction vector-valued wavelet packets are investigated. At last, the direct decomposition relation for space L₂(R)^r is presented.     

A two-parameter exact penalty function for nonlinear programming

A sequential quadratic programming algorithm for nonlinear programs using anl -exact penalty function is described. Numerical results are also presented. These results show that the algorithm is competitive with other exact penalty function based algorithms and that the inclusion of the second penalty parameter can be advantageous.     

Directional Haar wavelet frames on triangles

Traditional wavelets are not very effective in dealing with images that contain orientated discontinuities (edges). To achieve a more efficient representation one has to use basis elements with much higher directional sensitivity. In recent years several approaches like curvelets and shearlets have been studied providing essentially optimal approximation properties for images that are piecewise smooth and have discontinuities along C²-curves. While curvelets and shearlets have compact support in frequency domain, we construct directional wavelet frames generated by functions with compact support in time domain. Our Haar wavelet constructions can be seen as special composite dilation wavelets, being based on a generalized multiresolution analysis (MRA) associated with a dilation matrix and a finite collection of ‘shear’ matrices. The complete system of constructed wavelet functions forms a Parseval frame. Based on this MRA structure we provide an efficient filter bank algorithm. The freedom obtained by the redundancy of the applied Haar functions will be used for an efficient sparse representation of piecewise constant images as well as for image denoising.     

Curve denoising by multiscale singularity detection and geometric shrinkage

We propose a method for denoising piecewise smooth curves, given a number of noisy sample points. Using geometric variants of wavelet shrinkage methods, our algorithm preserves corners while enforcing that the smoothed arcs lie in an L² Sobolev space Hα of order α chosen by the operator. The reconstruction is scale-invariant when using the Sobolev space H3/2, adapts to the local noise level, and is essentially free of tuning parameters. In particular, our noise-adaptivity ensures that there is no arbitrarily-chosen “diffusion time” parameter in the denoising. Further, in cases where the distinction between signal and noise is unclear, we show how statistics gathered from the curve can be used to identify a finite number of “good” choices for the denoising.