Hilbertian convex feasibility problem: Convergence of projection methods

The classical problem of finding a point in the intersection of countably many closed and convex sets in a Hilbert space is considered. Extrapolated iterations of convex combinations of approximate projections onto subfamilies of sets are investigated to solve this problem. General hypotheses are made on the regularity of the sets and various strategies are considered to control the order in which the sets are selected. Weak and strong convergence results are established within thisbroad framework, which provides a unified view of projection methods for solving hilbertian convex feasibility problems. This work was supported by the National Science Foundation under Grant MIP-9308609.     

On the convergence of Han's method for convex programming with quadratic objective

We present an iterative method for minimizing strictly convex quadratic functions over the intersection of a finite number of convex sets. The method consists in computing projections onto the individual sets simultaneously and the new iterate is a convex combination of those projections. We give convergence proofs even for the inconsistent case, i.e. when the intersection of the sets is empty.Work of this author was partially supported by CNPq under grant No. 301280/86-MA.     

Projection and restriction methods in geometric programming and related problems

Mathematical programming problems with unattained infima or unbounded optimal solution sets are dual to problems which lackinterior points, e.g., problems for which the Slater condition fails to hold or for which the hypothesis of Fenchel's theorem fails to hold. In such cases, it is possible to project the unbounded problem onto a subspace and to restrict the dual problem to an affine set so that the infima are not altered. After a finite sequence of such projections and restrictions, dual problems are obtained which have bounded optimal solution sets andinterior points. Although results of this kind have occasionally been used in other contexts, it is in geometric programming (both in the original psynomial form and the generalized form) where such methods appear most useful. In this paper, we present a treatment of dual projection and restriction methods developed in terms of dual generalized geometric programming problems. Analogous results are given for Fenchel and ordinary dual problems.This research was supported in part by Grant No. AFOSR-73-2516 from the Air Force Office of Scientific Research and by Grant No. NSF-ENG-76-10260 from the National Science Foundation.The authors wish to express their appreciation to the referees for several helpful comments.     

Perturbed projections and subgradient projections for the multiple-sets split feasibility problem

We study the multiple-sets split feasibility problem that requires to find a point closest to a family of closed convex sets in one space such that its image under a linear transformation will be closest to another family of closed convex sets in the image space. By casting the problem into an equivalent problem in a suitable product space we are able to present a simultaneous subgradients projections algorithm that generates convergent sequences of iterates in the feasible case. We further derive and analyze a perturbed projection method for the multiple-sets split feasibility problem and, additionally, furnish alternative proofs to two known results.     

The rate of convergence for the cyclic projections algorithm III: Regularity of convex sets

The cyclic projections algorithm is an important method for determining a point in the intersection of a finite number of closed convex sets in a Hilbert space. That is, for determining a solution to the “convex feasibility” problem. This is the third paper in a series on a study of the rate of convergence for the cyclic projections algorithm. In the first of these papers, we showed that the rate could be described in terms of the “angles” between the convex sets involved. In the second, we showed that these angles often had a more tractable formulation in terms of the “norm” of the product of the (nonlinear) metric projections onto related convex sets.In this paper, we show that the rate of convergence of the cyclic projections algorithm is also intimately related to the “linear regularity property” of Bauschke and Borwein, the “normal property” of Jameson (as well as Bakan, Deutsch, and Li’s generalization of Jameson’s normal property), the “strong conical hull intersection property” of Deutsch, Li, and Ward, and the rate of convergence of iterated parallel projections. Such properties have already been shown to be important in various other contexts as well.     

Alternating convex projection methods for discrete-time covariance control design

The problem of designing a controller for a linear, discretetime system is formulated as a problem of designing an appropriate plant-state covariance matrix. Closed-loop stability and multiple-output performance constraints are expressed geometrically as requirements that the covariance matrix lies in the intersection of some specified closed, convex sets in the space of symmetric matrices. We solve a covariance feasibility problem to determine the existence and compute a covariance matrix to satisty assignability and output-norm performance constraints. In addition, we can treat a covariance optimization problem to construct an assignable covariance matrix which satisfies output performance constraints and is as close as possible to a given desired covariance. We can also treat inconsistent constraints, where we look for an assignable covariance which best approximates desired but unachievable output performance objectives; we call this the infeasible covariance optimization problem. All these problems are of a convex nature, and alternating convex projection methods are proposed to solve them, exploiting the geometric formulation of the problem. To this end, analytical expressions for the projections onto the covariance assignability and the output covariance inequality constraint sets are derived. Finally, the problem of designing low-order dynamic controllers using alternating projections is discussed, and a numerical technique using alternating projections is suggested for a solution, although convergence of the algorithm is not guaranteed in this case. A control design example for a fighter aircraft model illustrates the method.This research was completed while the first author was with the Space Systems Control Laboratory at Purdue University. Support from the Army Research Office Grant ARO-29029-EG is gratefully acknowledged.     

Alternating oblique projections for coupled linear systems

In this work we propose the use of alternating oblique projections (AOP) for the solution of the saddle points systems resulting from the discretization of domain decomposition problems. These systems are called coupled linear systems. The AOP method is a descent method in which the descent direction is defined by using alternating oblique projections onto the search subspaces. We prove that this method is a preconditioned simple gradient (Uzawa) method with a particular preconditioner. Finally, a preconditioned conjugate gradient based version of AOP is proposed. AMS subject classification 65F10, 65N22, 65Y05     

A note on the finite convergence of alternating projections

We establish sufficient conditions for finite convergence of the alternating projections method for two non-intersecting and potentially nonconvex sets. Our results are based on a generalization of the concept of intrinsic transversality, which until now has been restricted to sets with nonempty intersection. In the special case of a polyhedron and closed half space, our sufficient conditions define the minimum distance between the two sets that is required for alternating projections to converge in a single iteration.     

A multiprojection algorithm using Bregman projections in a product space

Generalized distances give rise to generalized projections into convex sets. An important question is whether or not one can use within the same projection algorithm different types of such generalized projections. This question has practical consequences in the area of signal detection and image recovery in situations that can be formulated mathematically as a convex feasibility problem. Using an extension of Pierra's product space formalism, we show here that a multiprojection algorithm converges. Our algorithm is fully simultaneous, i.e., it uses in each iterative stepall sets of the convex feasibility problem. Different multiprojection algorithms can be derived from our algorithmic scheme by a judicious choice of the Bregman functions which govern the process. As a by-product of our investigation we also obtain blockiterative schemes for certain kinds of linearly constraned optimization problems.     

A sequential iteration algorithm with non-monotoneous behaviour in the method of projections onto convex sets

The method of projections onto convex sets to find a point in the intersection of a finite number of closed convex sets in a Euclidean space, may lead to slow convergence of the constructed sequence when that sequence enters some narrow “corridor” between two or more convex sets. A way to leave such corridor consists in taking a big step at different moments during the iteration, because in that way the monotoneous behaviour that is responsible for the slow convergence may be interrupted. In this paper we present a technique that may introduce interruption of the monotony for a sequential algorithm, but that at the same time guarantees convergence of the constructed sequence to a point in the intersection of the sets. We compare experimentally the behaviour concerning the speed of convergence of the new algorithm with that of an existing monotoneous algorithm.     

A reduced-gradient variant of Karmarkar's algorithm and null-space projections

This paper discusses the relationship between Karmarkar's new method for linear programming and the traditional simplex method. It is shown how null-space Karmarkar projections can be done using a basis matrix to compute the projections in the null space. Preliminary computational evidence shows that problems exist in the choice of a basis matrix, but that, given a basis, very inexact and computationally efficient projections are computationally sound.     

The method of alternating projections and the method of subspace corrections in Hilbert space

A new identity is given in this paper for estimating the norm of the product of nonexpansive operators in Hilbert space. This identity can be applied for the design and analysis of the method of alternating projections and the method of subspace corrections. The method of alternating projections is an iterative algorithm for determining the best approximation to any given point in a Hilbert space from the intersection of a finite number of subspaces by alternatively computing the best approximations from the individual subspaces which make up the intersection. The method of subspace corrections is an iterative algorithm for finding the solution of a linear equation in a Hilbert space by approximately solving equations restricted on a number of closed subspaces which make up the entire space. The new identity given in the paper provides a sharpest possible estimate for the rate of convergence of these algorithms. It is also proved in the paper that the method of alternating projections is essentially equivalent to the method of subspace corrections. Some simple examples of multigrid and domain decomposition methods are given to illustrate the application of the new identity.

     

An interior points algorithm for the convex feasibility problem

The problem of finding a point in the intersection of a finite family of convex sets in the Euclidean space R″ is considered here. We present a general algorithmic scheme which employs projections onto separating hyperplanes instead of projections onto the convex sets. This scheme includes the method of successive projections of Gubin et al., USSR Comp. Math. and Math. Phys. 7 (1967), 1–24, as a special case. A different realization proposed here is capable of handling the problem when the sets are solid and an interior point of each set is available. This alternative algorithm may, in certain cases, be more attractive than the method of Gubin et al.     

Proximity Function Minimization Using Multiple Bregman Projections, with Applications to Split Feasibility and Kullback–Leibler Distance Minimization

Problems in signal detection and image recovery can sometimes be formulated as a convex feasibility problem (CFP) of finding a vector in the intersection of a finite family of closed convex sets. Algorithms for this purpose typically employ orthogonal or generalized projections onto the individual convex sets. The simultaneous multiprojection algorithm of Censor and Elfving for solving the CFP, in which different generalized projections may be used at the same time, has been shown to converge for the case of nonempty intersection; still open is the question of its convergence when the intersection of the closed convex sets is empty.Motivated by the geometric alternating minimization approach of Csiszár and Tusnády and the product space formulation of Pierra, we derive a new simultaneous multiprojection algorithm that employs generalized projections of Bregman to solve the convex feasibility problem or, in the inconsistent case, to minimize a proximity function that measures the average distance from a point to all convex sets. We assume that the Bregman distances involved are jointly convex, so that the proximity function itself is convex. When the intersection of the convex sets is empty, but the closure of the proximity function has a unique global minimizer, the sequence of iterates converges to this unique minimizer. Special cases of this algorithm include the Expectation Maximization Maximum Likelihood (EMML) method in emission tomography and a new convergence result for an algorithm that solves the split feasibility problem.     

Generalized Bregman Projections in Convex Feasibility Problems

We present a method for finding common points of finitely many closed convex sets in Euclidean space. The Bregman extension of the classical method of cyclic orthogonal projections employs nonorthogonal projections induced by a convex Bregman function, whereas the Bauschke and Borwein method uses Bregman/Legendre functions. Our method works with generalized Bregman functions (B-functions) and inexact projections, which are easier to compute than the exact ones employed in other methods. We also discuss subgradient algorithms with Bregman projections.     

Projection methods: an annotated bibliography of books and reviews

Projections onto sets are used in a wide variety of methods in optimization theory but not every method that uses projections really belongs to the class of projection methods as we mean it here. Here, projection methods are iterative algorithms that use projections onto sets while relying on the general principle that when a family of (usually closed and convex) sets is present, then projections (or approximate projections) onto the given individual sets are easier to perform than projections onto other sets (intersections, image sets under some transformation, etc.) that are derived from the given family of individual sets. Projection methods employ projections (or approximate projections) onto convex sets in various ways. They may use different kinds of projections and, sometimes, even use different projections within the same algorithm. They serve to solve a variety of problems which are either of the feasibility or the optimization types. They have different algorithmic structures, of which some are particularly suitable for parallel computing, and they demonstrate nice convergence properties and/or good initial behavioural patterns. This class of algorithms has witnessed great progress in recent years and its member algorithms have been applied with success to many scientific, technological and mathematical problems. This annotated bibliography includes books and review papers on, or related to, projection methods that we know about, use and like. If you know of books or review papers that should be added to this list please contact us.     

Linear and strong convergence of algorithms involving averaged nonexpansive operators

We introduce regularity notions for averaged nonexpansive operators. Combined with regularity notions of their fixed point sets, we obtain linear and strong convergence results for quasicyclic, cyclic, and random iterations. New convergence results on the Borwein–Tam method (BTM) and on the cyclically anchored Douglas–Rachford algorithm (CADRA) are also presented. Finally, we provide a numerical comparison of BTM, CADRA and the classical method of cyclic projections for solving convex feasibility problems.     

A relaxation method for minimizing a smooth function on a generalized spherical segment

The minimization of a smooth functional on a generalized spherical segment of a finite-dimensional Euclidean space is examined. A relaxation method that involves successive projections of the antigradient onto auxiliary sets of a simpler structure is proposed. It is shown that, under certain natural assumptions, this method converges to a stationary point.     

度量投影的收敛性（英文）

讨论了赋范空间中度量投影的收敛性．得到了在局部紧集控制下，Chebyshev凸集序列的度量投影的收敛性与K－M收敛，Wijsman收敛和Kuratowski收敛都等价．本文的结论完善了M．Tsukada在[1]和[2]结果．     

An iterative row-action method for interval convex programming

The iterative primal-dual method of Bregman for solving linearly constrained convex programming problems, which utilizes nonorthogonal projections onto hyperplanes, is represented in a compact form, and a complete proof of convergence is given for an almost cyclic control of the method. Based on this, a new algorithm for solving interval convex programming problems, i.e., problems of the form minf(x), subject to γ≤Ax≤δ, is proposed. For a certain family of functionsf(x), which includes the norm ∥x∥ and thex logx entropy function, convergence is proved. The present row-action method is particularly suitable for handling problems in which the matrixA is large (or huge) and sparse.