Steered sequential projections for the inconsistent convex feasibility problem

We study a steered sequential gradient algorithm which minimizes the sum of convex functions by proceeding cyclically in the directions of the negative gradients of the functions and using steered step-sizes. This algorithm is applied to the convex feasibility problem by minimizing a proximity function which measures the sum of the Bregman distances to the members of the family of convex sets. The resulting algorithm is a new steered sequential Bregman projection method which generates sequences that converge if they are bounded, regardless of whether the convex feasibility problem is or is not consistent. For orthogonal projections and affine sets the boundedness condition is always fulfilled.     

Dykstras algorithm with bregman projections: A convergence proof

Dykstra’s algorithm and the method of cyclic Bregman projections are often employed to solve best approximation and convex feasibility problems, which are fundamental in mathematics and the physical sciences. Censor and Reich very recently suggested a synthesis of these methods, Dykstra’s algorithm with Bregman projections, to tackle a non-orthogonal best approximation problem, They obtained convergence when each constraint is a halfspace. It is shown here that this new algorithm works for general closed convex constraints; this complements Censor and Reich’s result and relates to a framework by Tseng. The proof rests on Boyle and Dykstra’s original work and on strong properties of Bregman distances corresponding to Legendre functions. Special cases and observations simplifying the implementation of the algorithm are aiso discussed     

Proximity Function Minimization Using Multiple Bregman Projections, with Applications to Split Feasibility and Kullback–Leibler Distance Minimization

Problems in signal detection and image recovery can sometimes be formulated as a convex feasibility problem (CFP) of finding a vector in the intersection of a finite family of closed convex sets. Algorithms for this purpose typically employ orthogonal or generalized projections onto the individual convex sets. The simultaneous multiprojection algorithm of Censor and Elfving for solving the CFP, in which different generalized projections may be used at the same time, has been shown to converge for the case of nonempty intersection; still open is the question of its convergence when the intersection of the closed convex sets is empty.Motivated by the geometric alternating minimization approach of Csiszár and Tusnády and the product space formulation of Pierra, we derive a new simultaneous multiprojection algorithm that employs generalized projections of Bregman to solve the convex feasibility problem or, in the inconsistent case, to minimize a proximity function that measures the average distance from a point to all convex sets. We assume that the Bregman distances involved are jointly convex, so that the proximity function itself is convex. When the intersection of the convex sets is empty, but the closure of the proximity function has a unique global minimizer, the sequence of iterates converges to this unique minimizer. Special cases of this algorithm include the Expectation Maximization Maximum Likelihood (EMML) method in emission tomography and a new convergence result for an algorithm that solves the split feasibility problem.     

Some properties of generalized proximal point methods for quadratic and linear programming

The proximal point method for convex optimization has been extended recently through the use of generalized distances (e.g., Bregman distances) instead of the Euclidean one. One advantage of these extensions is the possibility of eliminating certain constraints (mainly positivity) from the subproblems, transforming an inequality constrained problem into a sequence of unconstrained or equality constrained problems. We consider here methods obtained using a certain class of Bregman functions applied to convex quadratic (including linear) programming, which are of the interior-point type. We prove that the limit of the sequence generated by the method lies in the relative interior of the solution set, and furthermore is the closest optimal solution to the initial point, in the sense of the Bregman distance. These results do not hold for the standard proximal point method, i.e., when the square of the Euclidean norm is used as the Bregman distance.The research leading to this paper was partially supported by CNPq Grant 301280/86.We thank two anonymous referees whose comments and suggestions allowed us to improve our results in a significant way.     

A multiprojection algorithm using Bregman projections in a product space

Generalized distances give rise to generalized projections into convex sets. An important question is whether or not one can use within the same projection algorithm different types of such generalized projections. This question has practical consequences in the area of signal detection and image recovery in situations that can be formulated mathematically as a convex feasibility problem. Using an extension of Pierra's product space formalism, we show here that a multiprojection algorithm converges. Our algorithm is fully simultaneous, i.e., it uses in each iterative stepall sets of the convex feasibility problem. Different multiprojection algorithms can be derived from our algorithmic scheme by a judicious choice of the Bregman functions which govern the process. As a by-product of our investigation we also obtain blockiterative schemes for certain kinds of linearly constraned optimization problems.     

An alternating projection that does not converge in norm

This paper proves that the method of alternating projections between two closed convex intersecting sets does not always converge in norm. Weak convergence was established by Bregman (Soviet Math. Dokl. 6 (1965) 688), but the status of norm convergence was undetermined. An explicit counterexample is provided.     

An interior point method with Bregman functions for the variational inequality problem with paramonotone operators

We present an algorithm for the variational inequality problem on convex sets with nonempty interior. The use of Bregman functions whose zone is the convex set allows for the generation of a sequence contained in the interior, without taking explicitly into account the constraints which define the convex set. We establish full convergence to a solution with minimal conditions upon the monotone operatorF, weaker than strong monotonicity or Lipschitz continuity, for instance, and including cases where the solution needs not be unique. We apply our algorithm to several relevant classes of convex sets, including orthants, boxes, polyhedra and balls, for which Bregman functions are presented which give rise to explicit iteration formulae, up to the determination of two scalar stepsizes, which can be found through finite search procedures. © 1998 The Mathematical Programming Society, Inc. Published by Elsevier Science B.V.Corresponding author.     

A Bregman projection method for approximating fixed points of quasi-Bregman nonexpansive mappings

We introduce an abstract algorithm that aims to find the Bregman projection onto a closed convex set. As an application, the asymptotic behavior of an iterative method for finding a fixed point of a quasi-Bregman nonexpansive mapping with the fixed-point closedness property is analyzed. We also show that our result is applicable to Bregman subgradient projectors.     

Convergence of Bregman Projection Methods for Solving Consistent Convex Feasibility Problems in Reflexive Banach Spaces

The problem that we consider is whether or under what conditions sequences generated in reflexive Banach spaces by cyclic Bregman projections on finitely many closed convex subsets Q _i with nonempty intersection converge to common points of the given sets.     

On the convergence of Han's method for convex programming with quadratic objective

We present an iterative method for minimizing strictly convex quadratic functions over the intersection of a finite number of convex sets. The method consists in computing projections onto the individual sets simultaneously and the new iterate is a convex combination of those projections. We give convergence proofs even for the inconsistent case, i.e. when the intersection of the sets is empty.Work of this author was partially supported by CNPq under grant No. 301280/86-MA.     

Bregman distances and Chebyshev sets

A closed set of a Euclidean space is said to be Chebyshev if every point in the space has one and only one closest point in the set. Although the situation is not settled in infinite-dimensional Hilbert spaces, in 1932 Bunt showed that in Euclidean spaces a closed set is Chebyshev if and only if the set is convex. In this paper, from the more general perspective of Bregman distances, we show that if every point in the space has a unique nearest point in a closed set, then the set is convex. We provide two approaches: one is by nonsmooth analysis; the other by maximal monotone operator theory. Subdifferentiability properties of Bregman nearest distance functions are also given.     

Re-examination of Bregman functions and new properties of their divergences

ABSTRACT

The Bregman divergence (Bregman distance, Bregman measure of distance) is a certain useful substitute for a distance, obtained from a well-chosen function (the ‘Bregman function’). Bregman functions and divergences have been extensively investigated during the last decades and have found applications in optimization, operations research, information theory, nonlinear analysis, machine learning and more. This paper re-examines various aspects related to the theory of Bregman functions and divergences. In particular, it presents many sufficient conditions which allow the construction of Bregman functions in a general setting and introduces new Bregman functions (such as a negative iterated log entropy). Moreover, it sheds new light on several known Bregman functions such as quadratic entropies, the negative Havrda-Charvát-Tsallis entropy, and the negative Boltzmann-Gibbs-Shannon entropy, and it shows that the negative Burg entropy, which is not a Bregman function according to the classical theory but nevertheless is known to have ‘Bregmanian properties’, can, by our re-examination of the theory, be considered as a Bregman function. Our analysis yields several by-products of independent interest such as the introduction of the concept of relative uniform convexity (a certain generalization of uniform convexity), new properties of uniformly and strongly convex functions, and results in Banach space theory.     

On a generalized proximal point method for solving equilibrium problems in Banach spaces

We introduce a regularized equilibrium problem in Banach spaces, involving generalized Bregman functions. For this regularized problem, we establish the existence and uniqueness of solutions. These regularizations yield a proximal-like method for solving equilibrium problems in Banach spaces. We prove that the proximal sequence is an asymptotically solving sequence when the dual space is uniformly convex. Moreover, we prove that all weak accumulation points are solutions if the equilibrium function is lower semicontinuous in its first variable. We prove, under additional assumptions, that the proximal sequence converges weakly to a solution.     

Nonlinear Proximal Decomposition Method for Convex Programming

In this paper, we propose a new decomposition method for solving convex programming problems with separable structure. The proposed method is based on the decomposition method proposed by Chen and Teboulle and the nonlinear proximal point algorithm using the Bregman function. An advantage of the proposed method is that, by a suitable choice of the Bregman function, each subproblem becomes essentially the unconstrained minimization of a finite-valued convex function. Under appropriate assumptions, the method is globally convergent to a solution of the problem.     

Equilibrium programming using proximal-like algorithms

We compute constrained equilibria satisfying an optimality condition. Important examples include convex programming, saddle problems, noncooperative games, and variational inequalities. Under a monotonicity hypothesis we show that equilibrium solutions can be found via iterative convex minimization. In the main algorithm each stage of computation requires two proximal steps, possibly using Bregman functions. One step serves to predict the next point; the other helps to correct the new prediction. To enhance practical applicability we tolerate numerical errors. Research supported partly by the Norwegian Research Council, project: Quantec 111039/401.     

Approximating a Zero of Sum of Two Monotone Operators Which Solves a Fixed Point Problem in Reflexive Banach Spaces

Abstract

The purpose of this paper is to introduce an iterative method for approximating a point in the set of zeros of the sum of two monotone mappings, which is also a solution of a fixed point problem for a Bregman strongly nonexpansive mapping in a real reflexive Banach space. With our iterative technique, we state and prove a strong convergence theorem for approximating an element in the intersection of the set of solutions of a variational inclusion problem for sum of two monotone mappings and the set of solutions of a fixed point problem for Bregman strongly nonexpansive mapping. We give applications of our result to convex minimization problem, convex feasibility problem, variational inequality problem, and equilibrium problem. Our result complements and extends some recent results in literature.     

Right Bregman nonexpansive operators in Banach spaces

We introduce and study new classes of Bregman nonexpansive operators in reflexive Banach spaces. These classes of operators are associated with the Bregman distance induced by a convex function. In particular, we characterize sunny right quasi-Bregman nonexpansive retractions, and as a consequence, we show that the fixed point set of any right quasi-Bregman nonexpansive operator is a sunny right quasi-Bregman nonexpansive retract of the ambient Banach space.     

An iterative row-action method for interval convex programming

The iterative primal-dual method of Bregman for solving linearly constrained convex programming problems, which utilizes nonorthogonal projections onto hyperplanes, is represented in a compact form, and a complete proof of convergence is given for an almost cyclic control of the method. Based on this, a new algorithm for solving interval convex programming problems, i.e., problems of the form minf(x), subject to γ≤Ax≤δ, is proposed. For a certain family of functionsf(x), which includes the norm ∥x∥ and thex logx entropy function, convergence is proved. The present row-action method is particularly suitable for handling problems in which the matrixA is large (or huge) and sparse.     

Relaxation Methods for Strictly Convex Regularizations of Piecewise Linear Programs

We give an algorithm for minimizing the sum of a strictly convex function and a convex piecewise linear function. It extends several dual coordinate ascent methods for large-scale linearly constrained problems that occur in entropy maximization, quadratic programming, and network flows. In particular, it may solve exact penalty versions of such (possibly inconsistent) problems, and subproblems of bundle methods for nondifferentiable optimization. It is simple, can exploit sparsity, and in certain cases is highly parallelizable. Its global convergence is established in the recent framework of B -functions (generalized Bregman functions). Accepted 29 October 1996     

Parallel iterative methods for solving systems of generalized mixed equilibrium problems in reflexive Banach spaces

The purpose of this paper is to introduce three parallel iterative methods which use techniques of Bregman distances, Bregman projections, Bregman strongly nonexpansive operators and hybrid or shrinking projection methods to solve systems of generalized mixed equilibrium problems in a real reflexive Banach space.