Subgradient Methods for Saddle-Point Problems

We study subgradient methods for computing the saddle points of a convex-concave function. Our motivation comes from networking applications where dual and primal-dual subgradient methods have attracted much attention in the design of decentralized network protocols. We first present a subgradient algorithm for generating approximate saddle points and provide per-iteration convergence rate estimates on the constructed solutions. We then focus on Lagrangian duality, where we consider a convex primal optimization problem and its Lagrangian dual problem, and generate approximate primal-dual optimal solutions as approximate saddle points of the Lagrangian function. We present a variation of our subgradient method under the Slater constraint qualification and provide stronger estimates on the convergence rate of the generated primal sequences. In particular, we provide bounds on the amount of feasibility violation and on the primal objective function values at the approximate solutions. Our algorithm is particularly well-suited for problems where the subgradient of the dual function cannot be evaluated easily (equivalently, the minimum of the Lagrangian function at a dual solution cannot be computed efficiently), thus impeding the use of dual subgradient methods.     

THE CENTRAL PATH IN SMOOTH CONVEX SEMIDEFINITE PROGRAMS

Abstract

In this paper we study the welldefinedness of the central path associated to a nonlinear convex semidefinite programming problem with smooth objective and constraint functions. Under standard assumptions, we prove that the existence of the central path is equivalent to the nonemptiness and boundedness of the optimal set. Other equivalent conditions are given, such as the existence of a strictly dual feasible point or the existence of a single central point. The monotonic behavior of the primal and dual logarithmic barriers and of the primal and dual objective functions along the trajectory is also discussed. The existence and optimality of cluster points is established and finally, under the additional assumption of analyticity of the data functions, the convergence of the primal-dual trajectory is proved.     

Universal duality in conic convex optimization

Given a primal-dual pair of linear programs, it is well known that if their optimal values are viewed as lying on the extended real line, then the duality gap is zero, unless both problems are infeasible, in which case the optimal values are +∞ and −∞. In contrast, for optimization problems over nonpolyhedral convex cones, a nonzero duality gap can exist when either the primal or the dual is feasible. For a pair of dual conic convex programs, we provide simple conditions on the ``constraint matrices' and cone under which the duality gap is zero for every choice of linear objective function and constraint right-hand side. We refer to this property as ``universal duality'. Our conditions possess the following properties: (i) they are necessary and sufficient, in the sense that if (and only if) they do not hold, the duality gap is nonzero for some linear objective function and constraint right-hand side; (ii) they are metrically and topologically generic; and (iii) they can be verified by solving a single conic convex program. We relate to universal duality the fact that the feasible sets of a primal convex program and its dual cannot both be bounded, unless they are both empty. Finally we illustrate our theory on a class of semidefinite programs that appear in control theory applications. This work was supported by a fellowship at the University of Maryland, in addition to NSF grants DEMO-9813057, DMI0422931, CUR0204084, and DoE grant DEFG0204ER25655. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation or those of the US Department of Energy.     

Simple bounds for solutions of monotone complementarity problems and convex programs

For a solvable monotone complementarity problem we show that each feasible point which is not a solution of the problem provides simple numerical bounds for some or all components of all solution vectors. Consequently for a solvable differentiable convex program each primal-dual feasible point which is not optimal provides simple bounds for some or all components of all primal-dual solution vectors. We also give an existence result and simple bounds for solutions of monotone compementarity problems satisfying a new, distributed constraint qualification. This result carries over to a simple existence and boundedness result for differentiable convex programs satisfying a similar constraint qualification.Sponsored by the United States Army under Contract No. DAAG29-80-C-0041. This material is based on work sponsored by National Science Foundation Grants MCS-8200632 and MCS-8102684.     

Vector duality for convex vector optimization problems by means of the quasi-interior of the ordering cone

We define weakly minimal elements of a set with respect to a convex cone by means of the quasi-interior of the cone and characterize them via linear scalarization, generalizing the classical weakly minimal elements from the literature. Then we attach to a general vector optimization problem, a dual vector optimization problem with respect to (generalized) weakly efficient solutions and establish new duality results. By considering particular cases of the primal vector optimization problem, we derive vector dual problems with respect to weakly efficient solutions for both constrained and unconstrained vector optimization problems and the corresponding weak, strong and converse duality statements.     

pth Power Lagrangian Method for Integer Programming

When does there exist an optimal generating Lagrangian multiplier vector (that generates an optimal solution of an integer programming problem in a Lagrangian relaxation formulation), and in cases of nonexistence, can we produce the existence in some other equivalent representation space? Under what conditions does there exist an optimal primal-dual pair in integer programming? This paper considers both questions. A theoretical characterization of the perturbation function in integer programming yields a new insight on the existence of an optimal generating Lagrangian multiplier vector, the existence of an optimal primal-dual pair, and the duality gap. The proposed pth power Lagrangian method convexifies the perturbation function and guarantees the existence of an optimal generating Lagrangian multiplier vector. A condition for the existence of an optimal primal-dual pair is given for the Lagrangian relaxation method to be successful in identifying an optimal solution of the primal problem via the maximization of the Lagrangian dual. The existence of an optimal primal-dual pair is assured for cases with a single Lagrangian constraint, while adopting the pth power Lagrangian method. This paper then shows that an integer programming problem with multiple constraints can be always converted into an equivalent form with a single surrogate constraint. Therefore, success of a dual search is guaranteed for a general class of finite integer programming problems with a prominent feature of a one-dimensional dual search.     

Conjugate maps and duality in multiobjective optimization

This paper considers duality in convex vector optimization. A vector optimization problem requires one to find all the efficient points of the attainable value set for given multiple objective functions. Embedding the primal problem into a family of perturbed problems enables one to define a dual problem in terms of the conjugate map of the perturbed objective function. Every solution of the stable primal problem is associated with a certain solution of the dual problem, which is characterized as a subgradient of the perturbed efficient value map. This pair of solutions also provides a saddle point of the Lagrangian map.     

Additively separable duality theory

In duality theory, there is a trade-off between generality and tractability. Thus, the generality of the Tind-Wolsey framework comes at the expense of an infinite-dimensional dual solution space, even if the primal solution space is finite dimensional. Therefore, the challenge is to impose additional structure on the dual solution space and to identify conditions on the primal program, such that the properties that are typically associated with duality, like weak and strong duality, are preserved.In this paper, we consider real-valuedness, continuity, and additive separability as such additional structures. The virtue of the latter property is that it restores the one-to-one correspondence between primal constraints and dual variables as it exists in Lagrangian duality. The main result of this paper is that, roughly speaking, the existence of realvalued, continuous, and additively separable dual solutions that preserve strong duality is guaranteed, once the primal program satisfies a certain stability condition. The latter condition is ensured by the well-known regularity conditions that imply constraint qualification in Karush-Kuhn-Tucker points. On the other hand, if instead of additive separability, a mild tractability condition is imposed on the dual solution space, then stability turns out to be a necessary condition for strong duality in a well-defined sense. This result, combined with the observation that applicability of some well-known augmented Lagrangian methods to constrained optimization.This study was supported by the Netherlands Foundation for Mathematics (SMC) with financial aid from the Netherlands Organization for Scientific Research (NWO).     

RAMP for the capacitated minimum spanning tree problem

This paper introduces dual and primal-dual RAMP algorithms for the solution of the capacitated minimum spanning tree problem (CMST). A surrogate constraint relaxation incorporating cutting planes is proposed to explore the dual solution space. In the dual RAMP approach, primal-feasible solutions are obtained by simple tabu searches that project dual solutions onto primal feasible space. A primal-dual approach is achieved by including a scatter search procedure that further exploits the adaptive memory framework. Computational results from applying the methods to a standard set of benchmark problems disclose that the dual RAMP algorithm finds high quality solutions very efficiently and that its primal-dual enhancement is still more effective.     

Posynomial geometric programming as a special case of semi-infinite linear programming

This paper develops a wholly linear formulation of the posynomial geometric programming problem. It is shown that the primal geometric programming problem is equivalent to a semi-infinite linear program, and the dual problem is equivalent to a generalized linear program. Furthermore, the duality results that are available for the traditionally defined primal-dual pair are readily obtained from the duality theory for semi-infinite linear programs. It is also shown that two efficient algorithms (one primal based and the other dual based) for geometric programming actually operate on the semi-infinite linear program and its dual.     

Conjugate Duality and the Control of Linear Discrete Systems

In this paper we deal with the minimization of a convex function over the solution set of a range inclusion problem determined by a multivalued operator with convex graph. We attach a dual problem to it, provide regularity conditions guaranteeing strong duality and derive for the resulting primal–dual pair necessary and sufficient optimality conditions. We also discuss the existence of optimal solutions for the primal and dual problems by using duality arguments. The theoretical results are applied in the context of the control of linear discrete systems.     

Towards Strong Duality in Integer Programming

We consider in this paper the Lagrangian dual method for solving general integer programming. New properties of Lagrangian duality are derived by a means of perturbation analysis. In particular, a necessary and sufficient condition for a primal optimal solution to be generated by the Lagrangian relaxation is obtained. The solution properties of Lagrangian relaxation problem are studied systematically. To overcome the difficulties caused by duality gap between the primal problem and the dual problem, we introduce an equivalent reformulation for the primal problem via applying a pth power to the constraints. We prove that this reformulation possesses an asymptotic strong duality property. Primal feasibility and primal optimality of the Lagrangian relaxation problems can be achieved in this reformulation when the parameter p is larger than a threshold value, thus ensuring the existence of an optimal primal-dual pair. We further show that duality gap for this partial pth power reformulation is a strictly decreasing function of p in the case of a single constraint. Dedicated to Professor Alex Rubinov on the occasion of his 65th birthday. Research supported by the Research Grants Council of Hong Kong under Grant CUHK 4214/01E, and the National Natural Science Foundation of China under Grants 79970107 and 10571116.     

Some characterizations of robust optimal solutions for uncertain convex optimization problems

In this paper, we consider robust optimal solutions for a convex optimization problem in the face of data uncertainty both in the objective and constraints. By using the properties of the subdifferential sum formulae, we first introduce a robust-type subdifferential constraint qualification, and then obtain some completely characterizations of the robust optimal solution of this uncertain convex optimization problem. We also investigate Wolfe type robust duality between the uncertain convex optimization problem and its uncertain dual problem by proving duality between the deterministic robust counterpart of the primal model and the optimistic counterpart of its dual problem. Moreover, we show that our results encompass as special cases some optimization problems considered in the recent literature.     

Fritz john second-order duality for nonlinear programming

A second-order dual to a nonlinear programming problem is formulated. This dual uses the Fritz John necessary optimality conditions instead of the Karush-Kuhn-Tucker necessary optimality conditions, and thus, does not require a constraint qualification. Weak, strong, strict-converse, and converse duality theorems between primal and dual problems are established.     

Approximating the nondominated set of an MOLP by approximately solving its dual problem

The geometric duality theory of Heyde and Löhne (2006) defines a dual to a multiple objective linear programme (MOLP). In objective space, the primal problem can be solved by Benson’s outer approximation method (Benson 1998a,b) while the dual problem can be solved by a dual variant of Benson’s algorithm (Ehrgott et al. 2007). Duality theory then assures that it is possible to find the (weakly) nondominated set of the primal MOLP by solving its dual. In this paper, we propose an algorithm to solve the dual MOLP approximately but within specified tolerance. This approximate solution set can be used to calculate an approximation of the weakly nondominated set of the primal. We show that this set is a weakly ε-nondominated set of the original primal MOLP and provide numerical evidence that this approach can be faster than solving the primal MOLP approximately.     

Duality in quasi-convex supremization and reverse convex infimization via abstract convex analysis,and applications to approximation **

We give duality theorems and dual characterizations of optimal solutions for abstract quasi-convex supremization problems and infimization problems with abstract reverse convex constraint sets. Our main tools are dualities between families of subsets, conjugations of type Lau associated to them, and subdifferentials with respect to conjugations of type Lau. These tools permit us to give explicitly the relation between.the constraint sets, and the relation between the objective functions, of the primal problem and the dual problem. As applications, we obtain duality theorems for quasi-convex supremization and reverse convex infimization in locally convex spaces and, in particular, for worst and best approximation in normed linear spaces.     

一类逼近l1精确罚函数的罚函数

本文对可微非线性规划问题提出了一个渐近算法,它是基于一类逼近l1精确罚函数的罚函数而提出的,我们证明了算法所得的极小点列的聚点均为原问题的最优解,并在Mangasarian-Fromovitz约束条件下,证明了有限次迭代之后,所有迭代均为可行的,即迭代所得的极小点为可行点.     

Robust Duality in Parametric Convex Optimization

Modelling of convex optimization in the face of data uncertainty often gives rise to families of parametric convex optimization problems. This motivates us to present, in this paper, a duality framework for a family of parametric convex optimization problems. By employing conjugate analysis, we present robust duality for the family of parametric problems by establishing strong duality between associated dual pair. We first show that robust duality holds whenever a constraint qualification holds. We then show that this constraint qualification is also necessary for robust duality in the sense that the constraint qualification holds if and only if robust duality holds for every linear perturbation of the objective function. As an application, we obtain a robust duality theorem for the best approximation problems with constraint data uncertainty under a strict feasibility condition.     

Sufficient optimality conditions and duality theory for interval optimization problem

This paper addresses the duality theory of a nonlinear optimization model whose objective function and constraints are interval valued functions. Sufficient optimality conditions are obtained for the existence of an efficient solution. Three type dual problems are introduced. Relations between the primal and different dual problems are derived. These theoretical developments are illustrated through numerical example.     

Robust duality for generalized convex programming problems under data uncertainty

In this paper we present a robust duality theory for generalized convex programming problems in the face of data uncertainty within the framework of robust optimization. We establish robust strong duality for an uncertain nonlinear programming primal problem and its uncertain Lagrangian dual by showing strong duality between the deterministic counterparts: robust counterpart of the primal model and the optimistic counterpart of its dual problem. A robust strong duality theorem is given whenever the Lagrangian function is convex. We provide classes of uncertain non-convex programming problems for which robust strong duality holds under a constraint qualification. In particular, we show that robust strong duality is guaranteed for non-convex quadratic programming problems with a single quadratic constraint with the spectral norm uncertainty under a generalized Slater condition. Numerical examples are given to illustrate the nature of robust duality for uncertain nonlinear programming problems. We further show that robust duality continues to hold under a weakened convexity condition.