Strong metric subregularity of mappings in variational analysis and optimization

Although the property of strong metric subregularity of set-valued mappings has been present in the literature under various names and with various (equivalent) definitions for more than two decades, it has attracted much less attention than its older “siblings”, the metric regularity and the strong (metric) regularity. The purpose of this paper is to show that the strong metric subregularity shares the main features of these two most popular regularity properties and is not less instrumental in applications. We show that the strong metric subregularity of a mapping F acting between metric spaces is stable under perturbations of the form $f+F$, where f is a function with a small calmness constant. This result is parallel to the Lyusternik–Graves theorem for metric regularity and to the Robinson theorem for strong regularity, where the perturbations are represented by a function f with a small Lipschitz constant. Then we study perturbation stability of the same kind for mappings acting between Banach spaces, where f is not necessarily differentiable but admits a set-valued derivative-like approximation. Strong metric q-subregularity is also considered, where q is a positive real constant appearing as exponent in the definition. Rockafellar's criterion for strong metric subregularity involving injectivity of the graphical derivative is extended to mappings acting in infinite-dimensional spaces. A sufficient condition for strong metric subregularity is established in terms of surjectivity of the Fréchet coderivative, and it is shown by a counterexample that surjectivity of the limiting coderivative is not a sufficient condition for this property, in general. Then various versions of Newton's method for solving generalized equations are considered including inexact and semismooth methods, for which superlinear convergence is shown under strong metric subregularity. As applications to optimization, a characterization of the strong metric subregularity of the KKT mapping is obtained, as well as a radius theorem for the optimality mapping of a nonlinear programming problem. Finally, an error estimate is derived for a discrete approximation in optimal control under strong metric subregularity of the mapping involved in the Pontryagin principle.     

Perturbing the Dual Feasible Region for Solving Convex Quadratic Programs

A dual l _p-norm perturbation approach is introduced for solving convex quadratic programming problems. The feasible region of the Lagrangian dual program is approximated by a proper subset that is defined by a single smooth convex constraint involving the l _p-norm of a vector measure of constraint violation. It is shown that the perturbed dual program becomes the dual program as p and, under some standard conditions, the optimal solution of the perturbed dual program converges to a dual optimal solution. A closed-form formula that converts an optimal solution of the perturbed dual program into a feasible solution of the primal convex quadratic program is also provided. Such primal feasible solutions converge to an optimal primal solution as p. The proposed approach generalizes the previously proposed primal perturbation approach with an entropic barrier function. Its theory specializes easily for linear programming.     

BCQ and Strong BCQ for Nonconvex Generalized Equations with Applications to Metric Subregularity

In this paper, based on basic constraint qualification (BCQ) and strong BCQ for convex generalized equation, we are inspired to further discuss constraint qualifications of BCQ and strong BCQ for nonconvex generalized equation and then establish their various characterizations. As applications, we use these constraint qualifications to study metric subregularity of nonconvex generalized equation and provide necessary and/or sufficient conditions in terms of constraint qualifications considered herein to ensure nonconvex generalized equation having metric subregularity.     

On Metric and Calmness Qualification Conditions in Subdifferential Calculus

The paper contains two groups of results. The first are criteria for calmness/subregularity for set-valued mappings between finite-dimensional spaces. We give a new sufficient condition whose subregularity part has the same form as the coderivative criterion for “full” metric regularity but involves a different type of coderivative which is introduced in the paper. We also show that the condition is necessary for mappings with convex graphs. The second group of results deals with the basic calculus rules of nonsmooth subdifferential calculus. For each of the rules we state two qualification conditions: one in terms of calmness/subregularity of certain set-valued mappings and the other as a metric estimate (not necessarily directly associated with aforementioned calmness/subregularity property). The conditions are shown to be weaker than the standard Mordukhovich–Rockafellar subdifferential qualification condition; in particular they cover the cases of convex polyhedral set-valued mappings and, more generally, mappings with semi-linear graphs. Relative strength of the conditions is thoroughly analyzed. We also show, for each of the calculus rules, that the standard qualification conditions are equivalent to “full” metric regularity of precisely the same mappings that are involved in the subregularity version of our calmness/subregularity condition. The research of Jiří V. Outrata was supported by the grant A 107 5402 of the Grant Agency of the Academy of Sciences of the Czech Republic.     

New algorithms for convex cost tension problem with application to computer vision

Motivated by various applications to computer vision, we consider the convex cost tension problem, which is the dual of the convex cost flow problem. In this paper, we first propose a primal algorithm for computing an optimal solution of the problem. Our primal algorithm iteratively updates primal variables by solving associated minimum cut problems. We show that the time complexity of the primal algorithm is $O(K?T(n,m))$, where $K$ is the range of primal variables and $T(n,m)$ is the time needed to compute a minimum cut in a graph with $n$ nodes and $m$ edges. We then develop an improved version of the primal algorithm, called the primal–dual algorithm, by making good use of dual variables in addition to primal variables. Although its time complexity is the same as that of the primal algorithm, we can expect a better performance in practice. We finally consider an application to a computer vision problem called the panoramic image stitching.     

Convex composite functions in Banach spaces and the primal lower-nice property

Primal lower-nice functions defined on Hilbert spaces provide examples of functions that are ``integrable' (i.e. of functions that are determined up to an additive constant by their subgradients). The class of primal lower-nice functions contains all convex and lower- functions. In finite dimensions the class of primal lower-nice functions also contains the composition of a convex function with a mapping under a constraint qualification. In Banach spaces certain convex composite functions were known to be primal lower-nice (e.g. a convex function had to be continuous relative to its domain). In this paper we weaken the assumptions and provide new examples of convex composite functions defined on a Banach space with the primal lower-nice property. One consequence of our results is the identification of new examples of integrable functions on Hilbert spaces.

     

Exactness conditions for an SDP relaxation of the extended trust region problem

In this paper we show that a standard SDP relaxation for so called extended trust-region problem is equivalent to a convex quadratic problem, with a linear objective and constraint functions and some additional simple convex quadratic constraints. Through this equivalence, new conditions, generalizing the ones existing in the literature, under which the SDP relaxation is exact, are introduced.     

On necessary conditions for a class of nondifferentiable minimax fractional programming

The Kuhn–Tucker-type necessary optimality conditions are given for the problem of minimizing a max fractional function, where the numerator of the function involved is the sum of a differentiable function and a convex function while the denominator is the difference of a differentiable function and a convex function, subject to a set of differentiable nonlinear inequalities on a convex subset C$C$ of Rⁿ${\mathbb{R}}^n$, under the conditions similar to the Kuhn–Tucker constraint qualification or the Arrow–Hurwicz–Uzawa constraint qualification or the Abadie constraint qualification. Relations with the calmness constraint qualification are given.     

Error bounds and metric subregularity

Necessary and sufficient criteria for metric subregularity (or calmness) of set-valued mappings between general metric or Banach spaces are treated in the framework of the theory of error bounds for a special family of extended real-valued functions of two variables. A classification scheme for the general error bound and metric subregularity criteria is presented. The criteria are formulated in terms of several kinds of primal and subdifferential slopes.     

Strongly convex weakly complex Berwald metrics and real Landsberg metrics

Under the assumption that' is a strongly convex weakly Khler Finsler metric on a complex manifold M, we prove that F is a weakly complex Berwald metric if and only if F is a real Landsberg metric.This result together with Zhong(2011) implies that among the strongly convex weakly Kahler Finsler metrics there does not exist unicorn metric in the sense of Bao(2007). We also give an explicit example of strongly convex Kahler Finsler metric which is simultaneously a complex Berwald metric, a complex Landsberg metric,a real Berwald metric, and a real Landsberg metric.     

Variational Analysis on Local Sharp Minima via Exact Penalization

In this paper we study local sharp minima of the nonlinear programming problem via exact penalization. Utilizing generalized differentiation tools in variational analysis such as subderivatives and regular subdifferentials, we obtain some primal and dual characterizations for a penalty function associated with the nonlinear programming problem to have a local sharp minimum. These general results are then applied to the ? _p penalty function with 0 ≤ p ≤ 1. In particular, we present primal and dual equivalent conditions in terms of the original data of the nonlinear programming problem, which guarantee that the ? _p penalty function has a local sharp minimum with a finite penalty parameter in the case of \(p\in (\frac {1}{2}, 1]\) and \(p=\frac {1}{2}\) respectively. By assuming the Guignard constraint qualification (resp. the generalized Guignard constraint qualification), we also show that a local sharp minimum of the nonlinear programming problem can be an exact local sharp minimum of the ? _p penalty function with p ∈ [0, 1] (resp. \(p\in [0, \frac {1}{2}]\)). Finally, we give some formulas for calculating the smallest penalty parameter for a penalty function to have a local sharp minimum.     

Metric subregularity for composite-convex generalized equations in Banach spaces

In this paper we consider a convex-composite generalized constraint equation in Banach spaces. Using variational analysis technique, in terms of normal cones and coderivatives, we first establish sufficient conditions for such an equation to be metrically subregular. Under the Robinson qualification, we prove that these conditions are also necessary for the metric subregularity. In particular, some existing results on error bound and metric subregularity are extended to the composite-convexity case from the convexity case.     

On proximinality of convex sets in superspaces

In this paper, we show that a closed convex subset C of a Banach space is strongly proximinal (proximinal, resp.) in every Banach space isometrically containing it if and only if C is locally (weakly, resp.) compact. As a consequence, it is proved that local compactness of C is also equivalent to that for every Banach space Y isometrically containing it, the metric projection from Y to C is nonempty set-valued and upper semi-continuous.     

ℓ1-Minimization with Magnitude Constraints in the Frequency Domain

In this paper, we study the -optimal control problem with additional constraints on the magnitude of the closed-loop frequency response. In particular, we study the case of magnitude constraints at fixed frequency points (a finite number of such constraints can be used to approximate an -norm constraint). In previous work, we have shown that the primal-dual formulation for this problem has no duality gap and both primal and dual problems are equivalent to convex, possibly infinite-dimensional, optimization problems with LMI constraints. Here, we study the effect of approximating the convex magnitude constraints with a finite number of linear constraints and provide a bound on the accuracy of the approximation. The resulting problems are linear programs. In the one-block case, both primal and dual programs are semi-infinite dimensional. The optimal cost can be approximated, arbitrarily well from above and within any predefined accuracy from below, by the solutions of finite-dimensional linear programs. In the multiblock case, the approximate LP problem (as well as the exact LMI problem) is infinite-dimensional in both the variables and the constraints. We show that the standard finite-dimensional approximation method, based on approximating the dual linear programming problem by sequences of finite-support problems, may fail to converge to the optimal cost of the infinite-dimensional problem.     

On Metric Regularity and the Boundary of the Feasible Set in Linear Optimization

This paper deals with semi-infinite linear inequality systems in ? ⁿ and studies the stability of the boundary of their feasible sets. We analyze the equivalence between the metric regularity of the inverse of the boundary set mapping, $\mathcal{N}$ , and the stability of the feasible set mapping in the sense of the maintenance of the consistency. In doing this we provide operational formulae for distances from points to some useful sets. We also include relationships between the regularity moduli corresponding to the mappings $\mathcal{N}$ and the inverse, $\mathcal{M}$ , of the feasible set mapping, and prove their equality for finite systems and some special cases in the semi-infinite framework. Moreover, we provide conditions to assure that the metric regularity of $\mathcal{N}$ is equivalent to the lower semi-continuity of the boundary set mapping, which is important because the latter property has many characterizations. Since the boundary of a feasible set may not be convex, we cannot make use of the general theory for mappings with convex graph, as for example, the Robinson–Ursescu theorem.     

Constraint qualifications in a class of nondifferentiable mathematical programming problems

The Kuhn-Tucker type necessary optimality conditions are given for the problem of minimizing the sum of a differentiable function and a convex function subject to a set of differentiable nonlinear inequalities on a convex subset C of , under the conditions similar to the Kuhn-Tucker constraint qualification or the Arrow-Hurwicz-Uzawa constraint qualification. The case when the set C is open (not necessarily convex) is shown to be a special one of our results, which helps us to improve some of the existing results in the literature.     

Theorems of the Alternative and Duality

This paper investigates the relations between theorems of the alternative and the minimum norm duality theorem. A typical theorem of the alternative is associated with two systems of linear inequalities and/or equalities, a primal system and a dual one, asserting that either the primal system has a solution, or the dual system has a solution, but never both. On the other hand, the minimum norm duality theorem says that the minimum distance from a given point z to a convex set is equal to the maximum of the distances from z to the hyperplanes separating z and . We consider the theorems of Farkas, Gale, Gordan, and Motzkin, as well as new theorems that characterize the optimality conditions of discrete l ₁-approximation problems and multifacility location problems. It is shown that, with proper choices of , each of these theorems can be recast as a pair of dual problems: a primal steepest descent problem that resembles the original primal system, and a dual least–norm problem that resembles the original dual system. The norm that defines the least-norm problem is the dual norm with respect to that which defines the steepest descent problem. Moreover, let y solve the least norm problem and let r denote the corresponding residual vector. If r=0, which means that z , then y solves the dual system. Otherwise, when r0 and z , any dual vector of r solves both the steepest descent problem and the primal system. In other words, let x solve the steepest descent problem; then, r and x are aligned. These results hold for any norm on . If the norm is smooth and strictly convex, then there are explicit rules for retrieving x from r and vice versa.     

Dual subgradient algorithms for large-scale nonsmooth learning problems

“Classical” First Order (FO) algorithms of convex optimization, such as Mirror Descent algorithm or Nesterov’s optimal algorithm of smooth convex optimization, are well known to have optimal (theoretical) complexity estimates which do not depend on the problem dimension. However, to attain the optimality, the domain of the problem should admit a “good proximal setup”. The latter essentially means that (1) the problem domain should satisfy certain geometric conditions of “favorable geometry”, and (2) the practical use of these methods is conditioned by our ability to compute at a moderate cost proximal transformation at each iteration. More often than not these two conditions are satisfied in optimization problems arising in computational learning, what explains why proximal type FO methods recently became methods of choice when solving various learning problems. Yet, they meet their limits in several important problems such as multi-task learning with large number of tasks, where the problem domain does not exhibit favorable geometry, and learning and matrix completion problems with nuclear norm constraint, when the numerical cost of computing proximal transformation becomes prohibitive in large-scale problems. We propose a novel approach to solving nonsmooth optimization problems arising in learning applications where Fenchel-type representation of the objective function is available. The approach is based on applying FO algorithms to the dual problem and using the accuracy certificates supplied by the method to recover the primal solution. While suboptimal in terms of accuracy guaranties, the proposed approach does not rely upon “good proximal setup” for the primal problem but requires the problem domain to admit a Linear Optimization oracle—the ability to efficiently maximize a linear form on the domain of the primal problem.     

THE CENTRAL PATH IN SMOOTH CONVEX SEMIDEFINITE PROGRAMS

Abstract

In this paper we study the welldefinedness of the central path associated to a nonlinear convex semidefinite programming problem with smooth objective and constraint functions. Under standard assumptions, we prove that the existence of the central path is equivalent to the nonemptiness and boundedness of the optimal set. Other equivalent conditions are given, such as the existence of a strictly dual feasible point or the existence of a single central point. The monotonic behavior of the primal and dual logarithmic barriers and of the primal and dual objective functions along the trajectory is also discussed. The existence and optimality of cluster points is established and finally, under the additional assumption of analyticity of the data functions, the convergence of the primal-dual trajectory is proved.     

A distributed Douglas-Rachford splitting method for multi-block convex minimization problems

A customized Douglas-Rachford splitting method (DRSM) was recently proposed to solve two-block separable convex optimization problems with linear constraints and simple abstract constraints. The algorithm has advantage over the well-known alternating direction method of multipliers (ADMM), the dual application of DRSM to the two-block convex minimization problem, in the sense that the subproblems can have larger opportunity of possessing closed-form solutions since they are unconstrained. In this paper, we further study along this way by considering the primal application of DRSM for the general case m≥3, i.e., we consider the multi-block separable convex minimization problem with linear constraints where the objective function is separable into m individual convex functions without coupled variables. The resulting method fully exploits the separable structure and enjoys decoupled subproblems which can be solved simultaneously. Both the exact and inexact versions of the new method are presented in a unified framework. Under mild conditions, we manage to prove the global convergence of the algorithm. Preliminary numerical experiments for extracting the background from corrupted surveillance video verify the encouraging efficiency of the new algorithm.