Fixed point and Bregman iterative methods for matrix rank minimization

The linearly constrained matrix rank minimization problem is widely applicable in many fields such as control, signal processing and system identification. The tightest convex relaxation of this problem is the linearly constrained nuclear norm minimization. Although the latter can be cast as a semidefinite programming problem, such an approach is computationally expensive to solve when the matrices are large. In this paper, we propose fixed point and Bregman iterative algorithms for solving the nuclear norm minimization problem and prove convergence of the first of these algorithms. By using a homotopy approach together with an approximate singular value decomposition procedure, we get a very fast, robust and powerful algorithm, which we call FPCA (Fixed Point Continuation with Approximate SVD), that can solve very large matrix rank minimization problems (the code can be downloaded from http://www.columbia.edu/~sm2756/FPCA.htm for non-commercial use). Our numerical results on randomly generated and real matrix completion problems demonstrate that this algorithm is much faster and provides much better recoverability than semidefinite programming solvers such as SDPT3. For example, our algorithm can recover 1000 × 1000 matrices of rank 50 with a relative error of 10^?5 in about 3?min by sampling only 20% of the elements. We know of no other method that achieves as good recoverability. Numerical experiments on online recommendation, DNA microarray data set and image inpainting problems demonstrate the effectiveness of our algorithms.     

Restricted linear constrained minimization of quadratic functionals

In this work, a linearly constrained minimization of a positive semidefinite quadratic functional is examined. We propose two different approaches to this problem. Our results are concerning infinite dimensional real Hilbert spaces, with a singular positive semidefinite operator related to the functional, and considering as constraint a singular operator. The difference between the proposed approaches for the minimization and previous work on this problem is that it is considered for all vectors belonging to the least squares solutions set, or to the vectors perpendicular to the kernel of the related operator or matrix.     

New reformulation and feasible semismooth Newton method for a class of stochastic linear complementarity problems

A class of stochastic linear complementarity problems (SLCPs) with finitely many realizations is considered. We first formulate the problem as a new constrained minimization problem. Then, we propose a feasible semismooth Newton method which yields a stationary point of the constrained minimization problem. We study the condition for the level set of the objective function to be bounded. As a result, the condition for the solution set of the constrained minimization problem is obtained. The global and quadratic convergence of the proposed method is proved under certain assumptions. Preliminary numerical results show that this method yields a reasonable solution with high safety and within a small number of iterations.     

Some iterative methods for finding fixed points and for solving constrained convex minimization problems

The present paper is divided into two parts. In the first part, we introduce implicit and explicit iterative schemes for finding the fixed point of a nonexpansive mapping defined on the closed convex subset of a real Hilbert space. We establish results on the strong convergence of the sequences generated by the proposed schemes to a fixed point of a nonexpansive mapping. Such a fixed point is also a solution of a variational inequality defined on the set of fixed points. In the second part, we propose implicit and explicit iterative schemes for finding the approximate minimizer of a constrained convex minimization problem and prove that the sequences generated by our schemes converge strongly to a solution of the constrained convex minimization problem. Such a solution is also a solution of a variational inequality defined over the set of fixed points of a nonexpansive mapping. The results of this paper extend and improve several results presented in the literature in the recent past.     

A Simply Constrained Optimization Reformulation of KKT Systems Arising from Variational Inequalities

The Karush—Kuhn—Tucker (KKT) conditions can be regarded as optimality conditions for both variational inequalities and constrained optimization problems. In order to overcome some drawbacks of recently proposed reformulations of KKT systems, we propose casting KKT systems as a minimization problem with nonnegativity constraints on some of the variables. We prove that, under fairly mild assumptions, every stationary point of this constrained minimization problem is a solution of the KKT conditions. Based on this reformulation, a new algorithm for the solution of the KKT conditions is suggested and shown to have some strong global and local convergence properties. Accepted 10 December 1997     

Image recovery via total variation minimization and related problems

Summary. We study here a classical image denoising technique introduced by L. Rudin and S. Osher a few years ago, namely the constrained minimization of the total variation (TV) of the image. First, we give results of existence and uniqueness and prove the link between the constrained minimization problem and the minimization of an associated Lagrangian functional. Then we describe a relaxation method for computing the solution, and give a proof of convergence. After this, we explain why the TV-based model is well suited to the recovery of some images and not of others. We eventually propose an alternative approach whose purpose is to handle the minimization of the minimum of several convex functionals. We propose for instance a variant of the original TV minimization problem that handles correctly some situations where TV fails. Received December 21, 1995 / Revised version February 26, 1996     

On convergence analysis of a derivative-free trust region algorithm for constrained optimization with separable structure

In this paper,we propose a derivative-free trust region algorithm for constrained minimization problems with separable structure,where derivatives of the objective function are not available and cannot be directly approximated.At each iteration,we construct a quadratic interpolation model of the objective function around the current iterate.The new iterates are generated by minimizing the augmented Lagrangian function of this model over the trust region.The filter technique is used to ensure the feasibility and optimality of the iterative sequence.Global convergence of the proposed algorithm is proved under some suitable assumptions.     

On the role of continuously differentiable exact penalty functions in constrained global optimization

The aim of this paper is to show that the new continuously differentiable exact penalty functions recently proposed in literature can play an important role in the field of constrained global optimization. In fact they allow us to transfer ideas and results proposed in unconstrained global optimization to the constrained case.First, by drawing our inspiration from the unconstrained case and by using the strong exactness properties of a particular continuously differentiable penalty function, we propose a sufficient condition for a local constrained minimum point to be global.Then we show that every constrained local minimum point satisfying the second order sufficient conditions is an attraction point for a particular implementable minimization algorithm based on the considered penalty function. This result can be used to define new classes of global algorithms for the solution of general constrained global minimization problems. As an example, in this paper we describe a simulated annealing algorithm which produces a sequence of points converging in probability to a global minimum of the original constrained problem.     

Splitting and linearizing augmented Lagrangian algorithm for subspace recovery from corrupted observations

Given a set of corrupted data drawn from a union of multiple subspace, the subspace recovery problem is to segment the data into their respective subspace and to correct the possible noise simultaneously. Recently, it is discovered that the task can be characterized, both theoretically and numerically, by solving a matrix nuclear-norm and a ?_2,1-mixed norm involved convex minimization problems. The minimization model actually has separable structure in both the objective function and constraint; it thus falls into the framework of the augmented Lagrangian alternating direction approach. In this paper, we propose and investigate an augmented Lagrangian algorithm. We split the augmented Lagrangian function and minimize the subproblems alternatively with one variable by fixing the other one. Moreover, we linearize the subproblem and add a proximal point term to easily derive the closed-form solutions. Global convergence of the proposed algorithm is established under some technical conditions. Extensive experiments on the simulated and the real data verify that the proposed method is very effective and faster than the sate-of-the-art algorithm LRR.     

On the Existence of Solitary Traveling Waves for Generalized Hertzian Chains

We consider the question of existence of ??bell-shaped?? (i.e., nonincreasing for x>0 and nondecreasing for x<0) traveling waves for the strain variable of the generalized Hertzian model describing, in the special case of a p=3/2 exponent, the dynamics of a granular chain. The proof of existence of such waves is based on the English and Pego (Proc. Am. Math. Soc. 133:1763, 2005) formulation of the problem. More specifically, we construct an appropriate energy functional, for which we show that the constrained minimization problem over bell-shaped entries has a solution. We also provide an alternative proof of the Friesecke?CWattis result (Commun. Math. Phys. 161:391, 1994) by using the same approach (but where the minimization is not constrained over bell-shaped curves). We briefly discuss and illustrate numerically the implications on the doubly exponential decay properties of the waves, as well as touch upon the modifications of these properties in the presence of a finite precompression force in the model.     

Bayesian comparison of models with inequality and equality constraints

A Bayesian model selection procedure for comparing models subject to inequality and/or equality constraints is proposed. An encompassing prior approach is used, and a general form of the Bayes factor of a constrained model against the encompassing model is derived. A simple estimation method is proposed which can estimate the Bayes factors for all candidate models simultaneously by using one set of samples from the encompassing model. A simulation study and a real data analysis demonstrate performance of the method.     

A VIF-based optimization model to alleviate collinearity problems in multiple linear regression

In this paper, we address data collinearity problems in multiple linear regression from an optimization perspective. We propose a novel linearly constrained quadratic programming model, based on the concept of the variance inflation factor (VIF). We employ the perturbation method that involves imposing a general symmetric non-diagonal perturbation matrix on the correlation matrix. The proposed VIF-based model reduces the largest VIF by minimizing the resulting biases. The VIF-based model can mitigate the harm from data collinearity through the reduction in both the condition number and VIFs, meanwhile improving the statistical significance. The resulting estimator has bounded biases under an iterative framework and hence is termed the least accumulative bias estimator. Certain potential statistical properties can be further considered as the side constraints for the proposed model. Various numerical examples validate the proposed approach.     

战斗对策:有约束极小极大途径(Ⅱ)

本文提出几个用集合,测度和积分工具来的极小极大判别准则,用来解有约束极小极大问题．     

An accelerated augmented Lagrangian method for linearly constrained convex programming with the rate of convergence <Emphasis Type="Italic">O</Emphasis>(1/<Emphasis Type="Italic">k</Emphasis><Superscript>2</Superscript>)

In this paper, we propose and analyze an accelerated augmented Lagrangian method (denoted by AALM) for solving the linearly constrained convex programming. We show that the convergence rate of AALM is O(1/k ²) while the convergence rate of the classical augmented Lagrangian method (ALM) is O(1/k). Numerical experiments on the linearly constrained l ₁?l ₂ minimization problem are presented to demonstrate the effectiveness of AALM.     

Non-convex and non-smooth variational decomposition for image restoration

The variational image decomposition model decomposes an image into a structural and an oscillatory component by regularization technique and functional minimization. It is an important task in various image processing methods, such as image restoration, image segmentation, and object recognition. In this paper, we propose a non-convex and non-smooth variational decomposition model for image restoration that uses non-convex and non-smooth total variation (TV) to measure the structure component and the negative Sobolev space $H^{-1}$ to model the oscillatory component. The new model combines the advantages of non-convex regularization and weaker-norm texture modeling, and it can well remove the noises while preserving the valuable edges and contours of the image. The iteratively reweighted l¹ (IRL1) algorithm is employed to solve the proposed non-convex minimization problem. For each subproblem, we use the alternating direction method of multipliers (ADMM) algorithm to solve it. Numerical results validate the effectiveness of the proposed model for both synthetic and real images in terms of peak signal-to-noise ratio (PSNR) and mean structural similarity index (MSSIM).     

A primal-dual active-set algorithm for bilaterally constrained total variation deblurring and piecewise constant Mumford-Shah segmentation problems

In this paper, we propose a fast primal-dual algorithm for solving bilaterally constrained total variation minimization problems which subsume the bilaterally constrained total variation image deblurring model and the two-phase piecewise constant Mumford-Shah image segmentation model. The presence of the bilateral constraints makes the optimality conditions of the primal-dual problem semi-smooth which can be solved by a semi-smooth Newton’s method superlinearly. But the linear system to solve at each iteration is very large and difficult to precondition. Using a primal-dual active-set strategy, we reduce the linear system to a much smaller and better structured one so that it can be solved efficiently by conjugate gradient with an approximate inverse preconditioner. Locally superlinear convergence results are derived for the proposed algorithm. Numerical experiments are also provided for both deblurring and segmentation problems. In particular, for the deblurring problem, we show that the addition of the bilateral constraints to the total variation model improves the quality of the solutions.     

Constrained and unconstrained rearrangement minimization problems related to the <Emphasis Type="Italic">p</Emphasis>-Laplace operator

In this paper we consider an unconstrained and a constrained minimization problem related to the boundary value problem

$$ - {\Delta _p}u = f{\text{ in }}D,{\text{ }}u = 0{\text{ on }}\partial D$$

. In the unconstrained problem we minimize an energy functional relative to a rearrangement class, and prove existence of a unique solution. We also consider the case when D is a planar disk and show that the minimizer is radial and increasing. In the constrained problem we minimize the energy functional relative to the intersection of a rearrangement class with an affine subspace of codimension one in an appropriate function space. We briefly discuss our motivation for studying the constrained minimization problem.

     

AN IMPLEMENTABLE ALGORITHM AND ITS CONVERGENCE FOR GLOBAL MINIMIZATION WITH CONSTRAINS

With the integral-level approach to global optimization, a class of discontinuous penalty functions is proposed to solve constrained minimization problems. In this paper we propose an implementable algorithm by means of the good point set of uniform distribution which conquers the default of Monte-Carlo method. At last we prove the convergence of the implementable algorithm.     

New Classes of Globally Convexized Filled Functions for Global Optimization

We propose new classes of globally convexized filled functions. Unlike the globally convexized filled functions previously proposed in literature, the ones proposed in this paper are continuously differentiable and, under suitable assumptions, their unconstrained minimization allows to escape from any local minima of the original objective function. Moreover we show that the properties of the proposed functions can be extended to the case of box constrained minimization problems. We also report the results of a preliminary numerical experience.     

A framelet algorithm for de-blurring images corrupted by multiplicative noise

This paper considers a variational model for restoring images from blurry and speckled observations. This model utilizes the favorable properties of framelet regularization (e.g., the sparsity and multiresolution properties of the framelet) that are well suited for speckle noise reduction. For solving the model, we first propose an approximation model that is motivated by the well-known variable-splitting and penalty techniques in optimization. We then develop an alternating minimization algorithm to solve the approximation model. We also show that the sequence generated by the algorithm converges to the solution of the proposed model. The numerical results on simulated data and real utrasound images demonstrate that our approach outperforms several state-of-the-art algorithms.