Generalized Bregman Projections in Convex Feasibility Problems

We present a method for finding common points of finitely many closed convex sets in Euclidean space. The Bregman extension of the classical method of cyclic orthogonal projections employs nonorthogonal projections induced by a convex Bregman function, whereas the Bauschke and Borwein method uses Bregman/Legendre functions. Our method works with generalized Bregman functions (B-functions) and inexact projections, which are easier to compute than the exact ones employed in other methods. We also discuss subgradient algorithms with Bregman projections.     

Steered sequential projections for the inconsistent convex feasibility problem

We study a steered sequential gradient algorithm which minimizes the sum of convex functions by proceeding cyclically in the directions of the negative gradients of the functions and using steered step-sizes. This algorithm is applied to the convex feasibility problem by minimizing a proximity function which measures the sum of the Bregman distances to the members of the family of convex sets. The resulting algorithm is a new steered sequential Bregman projection method which generates sequences that converge if they are bounded, regardless of whether the convex feasibility problem is or is not consistent. For orthogonal projections and affine sets the boundedness condition is always fulfilled.     

An extension of the proximal point algorithm with Bregman distances on Hadamard manifolds

In this paper we present an extension of the proximal point algorithm with Bregman distances to solve constrained minimization problems with quasiconvex and convex objective function on Hadamard manifolds. The proposed algorithm is a modified and extended version of the one presented in Papa Quiroz and Oliveira (J Convex Anal 16(1): 49–69, 2009). An advantage of the proposed algorithm, for the nonconvex case, is that in each iteration the algorithm only needs to find a stationary point of the proximal function and not a global minimum. For that reason, from the computational point of view, the proposed algorithm is more practical than the earlier proximal method. Another advantage, for the convex case, is that using minimal condition on the problem data as well as on the proximal parameters we get the same convergence results of the Euclidean proximal algorithm using Bregman distances.     

Construction of best Bregman approximations in reflexive Banach spaces

An iterative method is proposed to construct the Bregman projection of a point onto a countable intersection of closed convex sets in a reflexive Banach space.

     

A fast algorithm for image segmentation based on fuzzy region competition

Variational models for image segmentation are usually solved by the level set method, which is not only slow to compute but also dependent on initialization strongly. Recently, fuzzy region competition models or globally convex segmentation models have been introduced. They are insensitive to initialization, but contain TV-regularizers, making them difficult to compute. Goldstein, Bresson and Osher have applied the split Bregman iteration to globally convex segmentation models which avoided the regularization of TV norm and speeded up the computation. However, the split Bregman method needs to solve a partial differential equation (PDE) in each iteration. In this paper, we present a simple algorithm without solving the PDEs proposed originally by Jia et al. (2009) with application to image segmentation problems. The algorithm also avoids the regularization of TV norm and has a simpler form, which is in favor of implementing. Numerical experiments show that our algorithm works faster and more efficiently than other fast schemes, such as duality based methods and the split Bregman scheme.     

Re-examination of Bregman functions and new properties of their divergences

ABSTRACT

The Bregman divergence (Bregman distance, Bregman measure of distance) is a certain useful substitute for a distance, obtained from a well-chosen function (the ‘Bregman function’). Bregman functions and divergences have been extensively investigated during the last decades and have found applications in optimization, operations research, information theory, nonlinear analysis, machine learning and more. This paper re-examines various aspects related to the theory of Bregman functions and divergences. In particular, it presents many sufficient conditions which allow the construction of Bregman functions in a general setting and introduces new Bregman functions (such as a negative iterated log entropy). Moreover, it sheds new light on several known Bregman functions such as quadratic entropies, the negative Havrda-Charvát-Tsallis entropy, and the negative Boltzmann-Gibbs-Shannon entropy, and it shows that the negative Burg entropy, which is not a Bregman function according to the classical theory but nevertheless is known to have ‘Bregmanian properties’, can, by our re-examination of the theory, be considered as a Bregman function. Our analysis yields several by-products of independent interest such as the introduction of the concept of relative uniform convexity (a certain generalization of uniform convexity), new properties of uniformly and strongly convex functions, and results in Banach space theory.     

Right Bregman nonexpansive operators in Banach spaces

We introduce and study new classes of Bregman nonexpansive operators in reflexive Banach spaces. These classes of operators are associated with the Bregman distance induced by a convex function. In particular, we characterize sunny right quasi-Bregman nonexpansive retractions, and as a consequence, we show that the fixed point set of any right quasi-Bregman nonexpansive operator is a sunny right quasi-Bregman nonexpansive retract of the ambient Banach space.     

Some properties of generalized proximal point methods for quadratic and linear programming

The proximal point method for convex optimization has been extended recently through the use of generalized distances (e.g., Bregman distances) instead of the Euclidean one. One advantage of these extensions is the possibility of eliminating certain constraints (mainly positivity) from the subproblems, transforming an inequality constrained problem into a sequence of unconstrained or equality constrained problems. We consider here methods obtained using a certain class of Bregman functions applied to convex quadratic (including linear) programming, which are of the interior-point type. We prove that the limit of the sequence generated by the method lies in the relative interior of the solution set, and furthermore is the closest optimal solution to the initial point, in the sense of the Bregman distance. These results do not hold for the standard proximal point method, i.e., when the square of the Euclidean norm is used as the Bregman distance.The research leading to this paper was partially supported by CNPq Grant 301280/86.We thank two anonymous referees whose comments and suggestions allowed us to improve our results in a significant way.     

Proximity Function Minimization Using Multiple Bregman Projections, with Applications to Split Feasibility and Kullback–Leibler Distance Minimization

Problems in signal detection and image recovery can sometimes be formulated as a convex feasibility problem (CFP) of finding a vector in the intersection of a finite family of closed convex sets. Algorithms for this purpose typically employ orthogonal or generalized projections onto the individual convex sets. The simultaneous multiprojection algorithm of Censor and Elfving for solving the CFP, in which different generalized projections may be used at the same time, has been shown to converge for the case of nonempty intersection; still open is the question of its convergence when the intersection of the closed convex sets is empty.Motivated by the geometric alternating minimization approach of Csiszár and Tusnády and the product space formulation of Pierra, we derive a new simultaneous multiprojection algorithm that employs generalized projections of Bregman to solve the convex feasibility problem or, in the inconsistent case, to minimize a proximity function that measures the average distance from a point to all convex sets. We assume that the Bregman distances involved are jointly convex, so that the proximity function itself is convex. When the intersection of the convex sets is empty, but the closure of the proximity function has a unique global minimizer, the sequence of iterates converges to this unique minimizer. Special cases of this algorithm include the Expectation Maximization Maximum Likelihood (EMML) method in emission tomography and a new convergence result for an algorithm that solves the split feasibility problem.     

An interior proximal linearized method for DC programming based on Bregman distance or second-order homogeneous kernels

Abstract

We present an interior proximal method for solving constrained nonconvex optimization problems where the objective function is given by the difference of two convex function (DC function). To this end, we consider a linearized proximal method with a proximal distance as regularization. Convergence analysis of particular choices of the proximal distance as second-order homogeneous proximal distances and Bregman distances are considered. Finally, some academic numerical results are presented for a constrained DC problem and generalized Fermat–Weber location problems.     

A Bregman projection method for approximating fixed points of quasi-Bregman nonexpansive mappings

We introduce an abstract algorithm that aims to find the Bregman projection onto a closed convex set. As an application, the asymptotic behavior of an iterative method for finding a fixed point of a quasi-Bregman nonexpansive mapping with the fixed-point closedness property is analyzed. We also show that our result is applicable to Bregman subgradient projectors.     

Convergence analysis of the Bregman method for the variational model of image denoising

The total variation model of Rudin, Osher, and Fatemi for image denoising is considered to be one of the best denoising models. Recently, by using the Bregman method, Goldstein and Osher obtained a very efficient algorithm for the solution of the ROF model. In this paper, we give a rigorous proof for the convergence of the Bregman method. We also indicate that a combination of the Bregman method with wavelet packet decomposition often enhances performance for certain texture rich images.     

On Bregman-Type Distances for Convex Functions and Maximally Monotone Operators

Given two point to set operators, one of which is maximally monotone, we introduce a new distance in their graphs. This new concept reduces to the classical Bregman distance when both operators are the gradient of a convex function. We study the properties of this new distance and establish its continuity properties. We derive its formula for some particular cases, including the case in which both operators are linear monotone and continuous. We also characterize all bi-functions D for which there exists a convex function h such that D is the Bregman distance induced by h.     

Stochastic mirror descent method for distributed multi-agent optimization

This paper considers a distributed optimization problem encountered in a time-varying multi-agent network, where each agent has local access to its convex objective function, and cooperatively minimizes a sum of convex objective functions of the agents over the network. Based on the mirror descent method, we develop a distributed algorithm by utilizing the subgradient information with stochastic errors. We firstly analyze the effects of stochastic errors on the convergence of the algorithm and then provide an explicit bound on the convergence rate as a function of the error bound and number of iterations. Our results show that the algorithm asymptotically converges to the optimal value of the problem within an error level, when there are stochastic errors in the subgradient evaluations. The proposed algorithm can be viewed as a generalization of the distributed subgradient projection methods since it utilizes more general Bregman divergence instead of the Euclidean squared distance. Finally, some simulation results on a regularized hinge regression problem are presented to illustrate the effectiveness of the algorithm.     

Iterative Methods of Solving Stochastic Convex Feasibility Problems and Applications

The stochastic convex feasibility problem (SCFP) is the problem of finding almost common points of measurable families of closed convex subsets in reflexive and separable Banach spaces. In this paper we prove convergence criteria for two iterative algorithms devised to solve SCFPs. To do that, we first analyze the concepts of Bregman projection and Bregman function with emphasis on the properties of their local moduli of convexity. The areas of applicability of the algorithms we present include optimization problems, linear operator equations, inverse problems, etc., which can be represented as SCFPs and solved as such. Examples showing how these algorithms can be implemented are also given.     

An iterative row-action method for interval convex programming

The iterative primal-dual method of Bregman for solving linearly constrained convex programming problems, which utilizes nonorthogonal projections onto hyperplanes, is represented in a compact form, and a complete proof of convergence is given for an almost cyclic control of the method. Based on this, a new algorithm for solving interval convex programming problems, i.e., problems of the form minf(x), subject to γ≤Ax≤δ, is proposed. For a certain family of functionsf(x), which includes the norm ∥x∥ and thex logx entropy function, convergence is proved. The present row-action method is particularly suitable for handling problems in which the matrixA is large (or huge) and sparse.     

Epsilon-proximal decomposition method

We propose a modification of the proximal decomposition method investigated by Spingarn [30] and Mahey et al. [19] for minimizing a convex function on a subspace. For the method to be favorable from a computational point of view, particular importance is the introduction of approximations in the proximal step. First, we couple decomposition on the graph of the epsilon-subdifferential mapping and cutting plane approximations to get an algorithmic pattern that falls in the general framework of Rockafellar inexact proximal-point algorithms [26]. Recently, Solodov and Svaiter [27] proposed a new proximal point-like algorithm that uses improved error criteria and an enlargement of the maximal monotone operator defining the problem. We combine their idea with bundle mecanism to devise an inexact proximal decomposition method with error condition which is not hard to satisfy in practice. Then, we present some applications favorable to our development. First, we give a new regularized version of Benders decomposition method in convex programming called the proximal convex Benders decomposition algorithm. Second, we derive a new algorithm for nonlinear multicommodity flow problems among which the message routing problem in telecommunications data networks.     

Dykstras algorithm with bregman projections: A convergence proof

Dykstra’s algorithm and the method of cyclic Bregman projections are often employed to solve best approximation and convex feasibility problems, which are fundamental in mathematics and the physical sciences. Censor and Reich very recently suggested a synthesis of these methods, Dykstra’s algorithm with Bregman projections, to tackle a non-orthogonal best approximation problem, They obtained convergence when each constraint is a halfspace. It is shown here that this new algorithm works for general closed convex constraints; this complements Censor and Reich’s result and relates to a framework by Tseng. The proof rests on Boyle and Dykstra’s original work and on strong properties of Bregman distances corresponding to Legendre functions. Special cases and observations simplifying the implementation of the algorithm are aiso discussed     

A parallel descent algorithm for convex programming

In this paper, we propose a parallel decomposition algorithm for solving a class of convex optimization problems, which is broad enough to contain ordinary convex programming problems with a strongly convex objective function. The algorithm is a variant of the trust region method applied to the Fenchel dual of the given problem. We prove global convergence of the algorithm and report some computational experience with the proposed algorithm on the Connection Machine Model CM-5.     

Split Bregman method for the modified lot model in image denoising

In this paper a split Bregman iteration is proposed for the modified LOT model in image denoising. We first use the split Bregman method to solve the ROF model which can be seen as an approximate form of the first step of the original LOT model. Then we use a modified split Bregman method to fit the second step of the LOT model and give the convergence of the proposed split Bregman method. Several numerical examples are arranged to show the effectiveness of the proposed method.