带1-范数约束的分裂可行问题的投影算法

本文主要研究带1-范数约束的分裂可行问题的求解算法.用一种交替投影算法,求得了问题的解,提出松弛交替投影算法,改进了直接往闭凸集上投影这一不足,并证明了该算法的收敛性.     

An alternating projections method for certain linear problems in a Hilbert space

Recent extensions of von Neumann's alternating projection methodpermit the computation of proximity projections onto certainconvex sets. This paper exploits this fact in constructing aglobally convergent method for minimizing linear functions overa convex set in a Hilbert space. In particular, we solve theeducational testing problem and an inverse eigenvalue problem,two difficult problems involving positive semidefiniteness constraints.     

A finite steps algorithm for solving convex feasibility problems

This paper develops a new variant of the classical alternating projection method for solving convex feasibility problems where the constraints are given by the intersection of two convex cones in a Hilbert space. An extension to the feasibility problem for the intersection of two convex sets is presented as well. It is shown that one can solve such problems in a finite number of steps and an explicit upper bound for the required number of steps is obtained. As an application, we propose a new finite steps algorithm for linear programming with linear matrix inequality constraints. This solution is computed by solving a sequence of a matrix eigenvalue decompositions. Moreover, the proposed procedure takes advantage of the structure of the problem. In particular, it is well adapted for problems with several small size constraints.     

On the application of alternating projection under the constraint of linear matrix inequalities

In this paper, alternating projection under the constraint oflinear matrix inequalities (LMIs) is investigated to solve thefollowing two problems: finding the intersection of severalconvex LMI sets and designing an output-feedback stabilizingcontroller. Each problem is formulated as a quadratic optimizationproblem under LMI constraints. A numerical algorithm based onthe concept of alternating projection is proposed. The algorithmis demonstrated using a vertical-strip pole-assignment example.     

Deterministic global optimization with partition sets whose feasibility is not known: Application to concave minimization,reverse convex constraints,DC-programming,and Lipschitzian optimization

A crucial problem for many global optimization methods is how to handle partition sets whose feasibility is not known. This problem is solved for broad classes of feasible sets including convex sets, sets defined by finitely many convex and reverse convex constraints, and sets defined by Lipschitzian inequalities. Moreover, a fairly general theory of bounding is presented and applied to concave objective functions, to functions representable as differences of two convex functions, and to Lipschitzian functions. The resulting algorithms allow one to solve any global optimization problem whose objective function is of one of these forms and whose feasible set belongs to one of the above classes. In this way, several new fields of optimization are opened to the application of global methods.     

LMI Approach to Output Feedback Control for Linear Uncertain Systems with D-Stability Constraints

This paper deals with the problem of designing output feedback controllers for linear uncertain continuous-time and discrete-time systems with circular pole constraints. The uncertainty is assumed to be norm bounded and enters into both the system state and input matrices. We focus on the design of a dynamic output feedback controller that, for all admissible parameter uncertainties, assigns all the closed-loop poles inside a specified disk. It is shown that the problem addressed can be recast as a convex optimization problem characterized by linear matrix inequalities (LMI); therefore, an LMI approach is developed to derive the necessary and sufficient conditions for the existence of all desired dynamic output feedback controllers that achieve the specified circular pole constraints. An effective design procedure for the expected controllers is also presented. Finally, a numerical example is provided to show the usefulness and applicability of the present approach.     

Proximity Function Minimization Using Multiple Bregman Projections, with Applications to Split Feasibility and Kullback–Leibler Distance Minimization

Problems in signal detection and image recovery can sometimes be formulated as a convex feasibility problem (CFP) of finding a vector in the intersection of a finite family of closed convex sets. Algorithms for this purpose typically employ orthogonal or generalized projections onto the individual convex sets. The simultaneous multiprojection algorithm of Censor and Elfving for solving the CFP, in which different generalized projections may be used at the same time, has been shown to converge for the case of nonempty intersection; still open is the question of its convergence when the intersection of the closed convex sets is empty.Motivated by the geometric alternating minimization approach of Csiszár and Tusnády and the product space formulation of Pierra, we derive a new simultaneous multiprojection algorithm that employs generalized projections of Bregman to solve the convex feasibility problem or, in the inconsistent case, to minimize a proximity function that measures the average distance from a point to all convex sets. We assume that the Bregman distances involved are jointly convex, so that the proximity function itself is convex. When the intersection of the convex sets is empty, but the closure of the proximity function has a unique global minimizer, the sequence of iterates converges to this unique minimizer. Special cases of this algorithm include the Expectation Maximization Maximum Likelihood (EMML) method in emission tomography and a new convergence result for an algorithm that solves the split feasibility problem.     

Support vector machine classifiers with uncertain knowledge sets via robust optimization

In this article we study support vector machine (SVM) classifiers in the face of uncertain knowledge sets and show how data uncertainty in knowledge sets can be treated in SVM classification by employing robust optimization. We present knowledge-based SVM classifiers with uncertain knowledge sets using convex quadratic optimization duality. We show that the knowledge-based SVM, where prior knowledge is in the form of uncertain linear constraints, results in an uncertain convex optimization problem with a set containment constraint. Using a new extension of Farkas' lemma, we reformulate the robust counterpart of the uncertain convex optimization problem in the case of interval uncertainty as a convex quadratic optimization problem. We then reformulate the resulting convex optimization problems as a simple quadratic optimization problem with non-negativity constraints using the Lagrange duality. We obtain the solution of the converted problem by a fixed point iterative algorithm and establish the convergence of the algorithm. We finally present some preliminary results of our computational experiments of the method.     

Risk-Averse Stochastic Programming and Distributionally Robust Optimization Via Operator Splitting

This work deals with a broad class of convex optimization problems under uncertainty. The approach is to pose the original problem as one of finding a zero of the sum of two appropriate monotone operators, which is solved by the celebrated Douglas-Rachford splitting method. The resulting algorithm, suitable for risk-averse stochastic programs and distributionally robust optimization with fixed support, separates the random cost mapping from the risk function composing the problem’s objective. Such a separation is exploited to compute iterates by alternating projections onto different convex sets. Scenario subproblems, free from the risk function and thus parallelizable, are projections onto the cost mappings’ epigraphs. The risk function is handled in an independent and dedicated step consisting of evaluating its proximal mapping that, in many important cases, amounts to projecting onto a certain ambiguity set. Variables get updated by straightforward projections on subspaces through independent computations for the various scenarios. The investigated approach enjoys significant flexibility and opens the way to handle, in a single algorithm, several classes of risk measures and ambiguity sets.

     

交替方向法求解带线性约束的变分不等式

１引言变分不等式是一个有广泛应用的数学问题,它的一般形式是：确定一个向量,使其满足这里ｆ是一个从到自身的一个映射,Ｓ是Ｒ中的一个闭凸集．在许多实际问题中集合Ｓ往往具有如下结构其中ＡｂＫ是中的一个简单闭凸集．例如一个正卦限,一个框形约束结构,或者一个球简言之,Ｓ是Ｒ中的一个超平面与一个简单闭凸集的交．求解问题（１）－（２）,往往是通过对线性约束Ａ引人Ｌａｇｒａｎｇｅ乘子,将原问题化为如下的变分不等式：确定使得我们记问题（３）－（４）为ＶＩ（Ｆ）．熟知［３］,ＶＩ（,Ｆ）等价于投影方程其中凡（·）表…     

Multivariate Gaussians,semidefinite matrix completion,and convex algebraic geometry

We study multivariate normal models that are described by linear constraints on the inverse of the covariance matrix. Maximum likelihood estimation for such models leads to the problem of maximizing the determinant function over a spectrahedron, and to the problem of characterizing the image of the positive definite cone under an arbitrary linear projection. These problems at the interface of statistics and optimization are here examined from the perspective of convex algebraic geometry.     

Optimal kernel selection in twin support vector machines

In twin support vector machines (TWSVMs), we determine pair of non-parallel planes by solving two related SVM-type problems, each of which is smaller than the one in a conventional SVM. However, similar to other classification methods, the performance of the TWSVM classifier depends on the choice of the kernel. In this paper we treat the kernel selection problem for TWSVM as an optimization problem over the convex set of finitely many basic kernels, and formulate the same as an iterative alternating optimization problem. The efficacy of the proposed classification algorithm is demonstrated with some UCI machine learning benchmark datasets.     

Quasi-relative interior-type constraint qualifications ensuring strong Lagrange duality for optimization problems with cone and affine constraints

In this article we provide weak sufficient strong duality conditions for a convex optimization problem with cone and affine constraints, stated in infinite dimensional spaces, and its Lagrange dual problem. Our results are given by using the notions of quasi-relative interior and quasi-interior for convex sets. The main strong duality theorem is accompanied by several stronger, yet easier to verify in practice, versions of it. As exemplification we treat a problem which is inspired from network equilibrium. Our results come as corrections and improvements to Daniele and Giuffré (2007) [9].     

Projected Subgradient Minimization Versus Superiorization

The projected subgradient method for constrained minimization repeatedly interlaces subgradient steps for the objective function with projections onto the feasible region, which is the intersection of closed and convex constraints sets, to regain feasibility. The latter poses a computational difficulty, and, therefore, the projected subgradient method is applicable only when the feasible region is “simple to project onto.” In contrast to this, in the superiorization methodology a feasibility-seeking algorithm leads the overall process, and objective function steps are interlaced into it. This makes a difference because the feasibility-seeking algorithm employs projections onto the individual constraints sets and not onto the entire feasible region. We present the two approaches side-by-side and demonstrate their performance on a problem of computerized tomography image reconstruction, posed as a constrained minimization problem aiming at finding a constraint-compatible solution that has a reduced value of the total variation of the reconstructed image.     

Hilbertian convex feasibility problem: Convergence of projection methods

The classical problem of finding a point in the intersection of countably many closed and convex sets in a Hilbert space is considered. Extrapolated iterations of convex combinations of approximate projections onto subfamilies of sets are investigated to solve this problem. General hypotheses are made on the regularity of the sets and various strategies are considered to control the order in which the sets are selected. Weak and strong convergence results are established within thisbroad framework, which provides a unified view of projection methods for solving hilbertian convex feasibility problems. This work was supported by the National Science Foundation under Grant MIP-9308609.     

Restricted Normal Cones and Sparsity Optimization with Affine Constraints

The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely behaved nonconvex relaxations. In this paper we consider the elementary method of alternating projections (MAP) for solving the sparsity optimization problem without employing convex heuristics. In a parallel paper we recently introduced the restricted normal cone which generalizes the classical Mordukhovich normal cone and reconciles some fundamental gaps in the theory of sufficient conditions for local linear convergence of the MAP algorithm. We use the restricted normal cone together with the notion of superregularity, which is inherently satisfied for the affine sparse optimization problem, to obtain local linear convergence results with estimates for the radius of convergence of the MAP algorithm applied to sparsity optimization with an affine constraint.     

Projection methods: an annotated bibliography of books and reviews

Projections onto sets are used in a wide variety of methods in optimization theory but not every method that uses projections really belongs to the class of projection methods as we mean it here. Here, projection methods are iterative algorithms that use projections onto sets while relying on the general principle that when a family of (usually closed and convex) sets is present, then projections (or approximate projections) onto the given individual sets are easier to perform than projections onto other sets (intersections, image sets under some transformation, etc.) that are derived from the given family of individual sets. Projection methods employ projections (or approximate projections) onto convex sets in various ways. They may use different kinds of projections and, sometimes, even use different projections within the same algorithm. They serve to solve a variety of problems which are either of the feasibility or the optimization types. They have different algorithmic structures, of which some are particularly suitable for parallel computing, and they demonstrate nice convergence properties and/or good initial behavioural patterns. This class of algorithms has witnessed great progress in recent years and its member algorithms have been applied with success to many scientific, technological and mathematical problems. This annotated bibliography includes books and review papers on, or related to, projection methods that we know about, use and like. If you know of books or review papers that should be added to this list please contact us.     

Characterizing the solution set of convex optimization problems without convexity of constraints

Some characterizations of solution sets of a convex optimization problem with a convex feasible set described by tangentially convex constraints are given.     

一类自适应广义交替方向乘子法

广义交替方向乘子法是求解凸优化问题的有效算法.当实际问题中子问题难以求解时,可以采用在子问题中添加邻近项的方法处理,邻近矩阵正定时,算法收敛,然而这也会使迭代步长较小.最新研究表明,邻近矩阵可以有一定的不正定性.本文在基于不定邻近项的广义交替方向乘子法框架下,提出一种自适应的广义交替方向乘子法,动态地选择邻近矩阵,增大迭代步长.在一些较弱的假设下,证明了算法的全局收敛性.我们进行一些初等数值实验,验证了算法的有效性.     

Dykstra's Algorithm for a Constrained Least-squares Matrix Problem

We apply Dykstra's alternating projection algorithm to the constrained least-squares matrix problem that arises naturally in statistics and mathematical economics. In particular, we are concerned with the problem of finding the closest symmetric positive definite bounded and patterned matrix, in the Frobenius norm, to a given matrix. In this work, we state the problem as the minimization of a convex function over the intersection of a finite collection of closed and convex sets in the vector space of square matrices. We present iterative schemes that exploit the geometry of the problem, and for which we establish convergence to the unique solution. Finally, we present preliminary numberical results to illustrate the performance of the proposed iterative methods.