An active-set algorithm for norm constrained quadratic problems

Mathematical Programming - We present an algorithm for the minimization of a nonconvex quadratic function subject to linear inequality constraints and a two-sided bound on the 2-norm of its...     

A primal-dual interior-point algorithm for nonlinear least squares constrained problems

This paper extends prior work by the authors on solving nonlinear least squares unconstrained problems using a factorized quasi-Newton technique. With this aim we use a primal-dual interior-point algorithm for nonconvex nonlinear programming. The factorized quasi-Newton technique is now applied to the Hessian of the Lagrangian function for the transformed problem which is based on a logarithmic barrier formulation. We emphasize the importance of establishing and maintaining symmetric quasi-definiteness of the reduced KKT system. The algorithm then tries to choose a step size that reduces a merit function, and to select a penalty parameter that ensures descent directions along the iterative process. Computational results are included for a variety of least squares constrained problems and preliminary numerical testing indicates that the algorithm is robust and efficient in practice.     

A fast alternating minimization algorithm for total variation deblurring without boundary artifacts

Recently, a fast alternating minimization algorithm for total variation image deblurring (FTVd) has been presented by Wang, Yang, Yin, and Zhang (2008) [32]. The method in a nutshell consists of a discrete Fourier transform-based alternating minimization algorithm with periodic boundary conditions and in which two fast Fourier transforms (FFTs) are required per iteration. In this paper, we propose an alternating minimization algorithm for the continuous version of the total variation image deblurring problem. We establish convergence of the proposed continuous alternating minimization algorithm. The continuous setting is very useful to have a unifying representation of the algorithm, independently of the discrete approximation of the deconvolution problem, in particular concerning the strategies for dealing with boundary artifacts. Indeed, an accurate restoration of blurred and noisy images requires a proper treatment of the boundary. A discrete version of our continuous alternating minimization algorithm is obtained following two different strategies: the imposition of appropriate boundary conditions and the enlargement of the domain. The first one is computationally useful in the case of a symmetric blur, while the second one can be efficiently applied for a nonsymmetric blur. Numerical tests show that our algorithm generates higher quality images in comparable running times with respect to the Fast Total Variation deconvolution algorithm.     

Split Bregman iteration algorithm for total bounded variation regularization based image deblurring

Many existing algorithms taking the seminorm in BV(Ω) for regularization have achieved great success in image processing. However, this paper considers the total bounded variation regularization based approach to perform image deblurring. Based on this novel model, we introduce an extended split Bregman iteration to obtain the optimum solution quickly. We also provide the rigorous convergence analysis of the iterative algorithm here. Compared with the results of the ROF method, numerical simulations illustrate the more excellent reconstruction performance of the proposed algorithm.     

Study of a primal-dual algorithm for equality constrained minimization

The paper proposes a primal-dual algorithm for solving an equality constrained minimization problem. The algorithm is a Newton-like method applied to a sequence of perturbed optimality systems that follow naturally from the quadratic penalty approach. This work is first motivated by the fact that a primal-dual formulation of the quadratic penalty provides a better framework than the standard primal form. This is highlighted by strong convergence properties proved under standard assumptions. In particular, it is shown that the usual requirement of solving the penalty problem with a precision of the same size as the perturbation parameter, can be replaced by a much less stringent criterion, while guaranteeing the superlinear convergence property. A second motivation is that the method provides an appropriate regularization for degenerate problems with a rank deficient Jacobian of constraints. The numerical experiments clearly bear this out. Another important feature of our algorithm is that the penalty parameter is allowed to vary during the inner iterations, while it is usually kept constant. This alleviates the numerical problem due to ill-conditioning of the quadratic penalty, leading to an improvement of the numerical performances.     

A primal-dual approximation algorithm for generalized steiner network problems

We present the first polynomial-time approximation algorithm for finding a minimum-cost subgraph having at least a specified number of edges in each cut. This class of problems includes, among others, the generalized Steiner network problem, also called the survivable network design problem. Ifk is the maximum cut requirement of the problem, our solution comes within a factor of 2k of optimal. Our algorithm is primal-dual and shows the importance of this technique in designing approximation algorithms.Research supported by an NSF Graduate Fellowship, DARPA contracts N00014-91-J-1698 and N00014-92-J-1799, and AT&T Bell Laboratories.Research supported in part by Air Force contract F49620-92-J-0125 and DARPA contract N00014-92-J-1799.Part of this work was done while the author was visiting AT&T Bell Laboratories and Bellcore.     

A splitting primal-dual proximity algorithm for solving composite optimization problems

Our work considers the optimization of the sum of a non-smooth convex function and a finite family of composite convex functions, each one of which is composed of a convex function and a bounded linear operator. This type of problem is associated with many interesting challenges encountered in the image restoration and image reconstruction fields. We developed a splitting primal-dual proximity algorithm to solve this problem. Furthermore, we propose a preconditioned method, of which the iterative parameters are obtained without the need to know some particular operator norm in advance. Theoretical convergence theorems are presented. We then apply the proposed methods to solve a total variation regularization model, in which the L2 data error function is added to the L1 data error function. The main advantageous feature of this model is its capability to combine different loss functions. The numerical results obtained for computed tomography (CT) image reconstruction demonstrated the ability of the proposed algorithm to reconstruct an image with few and sparse projection views while maintaining the image quality.     

An automatic regularization parameter selection algorithm in the total variation model for image deblurring

Image restoration is an inverse problem that has been widely studied in recent years. The total variation based model by Rudin-Osher-Fatemi (1992) is one of the most effective and well known due to its ability to preserve sharp features in restoration. This paper addresses an important and yet outstanding issue for this model in selection of an optimal regularization parameter, for the case of image deblurring. We propose to compute the optimal regularization parameter along with the restored image in the same variational setting, by considering a Karush Kuhn Tucker (KKT) system. Through establishing analytically the monotonicity result, we can compute this parameter by an iterative algorithm for the KKT system. Such an approach corresponds to solving an equation using discrepancy principle, rather than using discrepancy principle only as a stopping criterion. Numerical experiments show that the algorithm is efficient and effective for image deblurring problems and yet is competitive.     

A Retinex modulated piecewise constant variational model for image segmentation and bias correction

In this paper, we propose a novel Retinex induced piecewise constant variational model for simultaneous segmentation of images with intensity inhomogeneity and bias correction. Firstly, we obtain an additive model by decomposing the original image into a smooth bias component and a structure part based on the Retinex theory. Secondly, the structure part can be modeled by the piecewise constant variational model and thus deduced a new data fidelity term. Finally, we formulate a new energy functional by incorporating the data fidelity term into the level set framework and introducing a GL-regularizer to the level set function and a smooth regularizer to model the bias component. Based on the alternating minimization algorithm and the operator splitting method, we present a numerical scheme to solve the minimization problem efficiently. Experimental results on images from diverse modalities demonstrate the competitive performances of the proposed model and algorithm over other representative methods in term of efficiency and robustness.     

A hybrid algorithm for linearly constrained minimax problems

Many real life problems can be stated as a minimax problem, such as economics, finance, management, engineering and other fields, which demonstrate the importance of having reliable methods to tackle minimax problems. In this paper, an algorithm for linearly constrained minimax problems is presented in which we combine the trust-region methods with the line-search methods and curve-search methods. By means of this hybrid technique, it avoids possibly solving the trust-region subproblems many times, and make better use of the advantages of different methods. Under weaker conditions, the global and superlinear convergence are achieved. Numerical experiments show that the new algorithm is robust and efficient.     

A derivative-free algorithm for linearly constrained optimization problems

Based on the NEWUOA algorithm, a new derivative-free algorithm is developed, named LCOBYQA. The main aim of the algorithm is to find a minimizer $x^{*} \in\mathbb{R}^{n}$ of a non-linear function, whose derivatives are unavailable, subject to linear inequality constraints. The algorithm is based on the model of the given function constructed from a set of interpolation points. LCOBYQA is iterative, at each iteration it constructs a quadratic approximation (model) of the objective function that satisfies interpolation conditions, and leaves some freedom in the model. The remaining freedom is resolved by minimizing the Frobenius norm of the change to the second derivative matrix of the model. The model is then minimized by a trust-region subproblem using the conjugate gradient method for a new iterate. At times the new iterate is found from a model iteration, designed to improve the geometry of the interpolation points. Numerical results are presented which show that LCOBYQA works well and is very competing against available model-based derivative-free algorithms.     

A recursive algorithm for constrained two-dimensional cutting problems

This paper presents a recursive algorithm for constrained two-dimensional guillotine cutting problems of rectangular items. The algorithm divides a stock plate into a sequence of small rectangular blocks. For the current block considered, it selects an item, puts it at the left-bottom corner of the block, and determines the direction of the dividing cut that divides the unoccupied region of the block into two smaller blocks for further consideration. The dividing cut is either along the upper edge or along the right edge of the selected item. The upper bound obtained from the unconstrained solution is used to shorten the searching space. The computational results on benchmark problems indicate that the algorithm can improve the solutions, and is faster than other algorithms.     

A primal-dual conjugate subgradient algorithm for specially structured linear and convex programming problems

This paper presents a primal-dual conjugate subgradient algorithm for solving convex programming problems. The motivation, however, is to employ it for solving specially structured or decomposable linear programming problems. The algorithm coordinates a primal penalty function and a Lagrangian dual function, in order to generate a (geometrically) convergent sequence of primal and dual iterates. Several refinements are discussed to improve the performance of the algorithm. These are tested on some network problems, with side constraints and variables, faced by the Freight Equipment Management Program of the Association of American Railroads, and suggestions are made for implementation.This research was supported by the Association of American Railroads.     

A new stochastic variational PDE model for soft Mumford-Shah segmentation

In this paper, a new stochastic variational PDE model is developed, using instead of hard segmentation soft segmentation. In this way, each pixel is allowed to belong to each image pattern with some probability. Our work proposes a functional with variable exponent, which provides a more accurate model for image segmentation and denoising. The diffusion resulting from the proposed model is a combination between TV-based and isotropic smoothing. The modeling procedure, computational implementation and results are explored in detail and numerical examples of real and synthetic images are presented.     

A feasible and superlinear algorithm for inequality constrained minimization problems

1. IntroductionConsider the optimization problemmin {f(x): gi(x) 5 0, j E I; x E R"}, (l)where f(x), gi(x): Rad - R, j E I ~ {1, 2,...,m}.It is well known that one of the most effective methods to solve problem (1) is thesequential quadratic programming (i.e., SoP) (see [1--6]), due to its property of superlinearconvergence. Especially in recent years, in order to perfect SoP both in theory and application, there have many papers, such as [7--10], been published. These papers focus mainly…