Discretization and localization in successive convex relaxation methods for nonconvex quadratic optimization

Based on the authors’ previous work which established theoretical foundations of two, conceptual, successive convex relaxation methods, i.e., the SSDP (Successive Semidefinite Programming) Relaxation Method and the SSILP (Successive Semi-Infinite Linear Programming) Relaxation Method, this paper proposes their implementable variants for general quadratic optimization problems. These problems have a linear objective function c ^T x to be maximized over a nonconvex compact feasible region F described by a finite number of quadratic inequalities. We introduce two new techniques, “discretization” and “localization,” into the SSDP and SSILP Relaxation Methods. The discretization technique makes it possible to approximate an infinite number of semi-infinite SDPs (or semi-infinite LPs) which appeared at each iteration of the original methods by a finite number of standard SDPs (or standard LPs) with a finite number of linear inequality constraints. We establish:?•Given any open convex set U containing F, there is an implementable discretization of the SSDP (or SSILP) Relaxation Method which generates a compact convex set C such that F⊆C⊆U in a finite number of iterations.?The localization technique is for the cases where we are only interested in upper bounds on the optimal objective value (for a fixed objective function vector c) but not in a global approximation of the convex hull of F. This technique allows us to generate a convex relaxation of F that is accurate only in certain directions in a neighborhood of the objective direction c. This cuts off redundant work to make the convex relaxation accurate in unnecessary directions. We establish:?•Given any positive number ε, there is an implementable localization-discretization of the SSDP (or SSILP) Relaxation Method which generates an upper bound of the objective value within ε of its maximum in a finite number of iterations. Received: June 30, 1998 / Accepted: May 18, 2000?Published online September 20, 2000     

On convergence of the Gauss-Newton method for convex composite optimization

The local quadratic convergence of the Gauss-Newton method for convex composite optimization f=h∘F is established for any convex function h with the minima set C, extending Burke and Ferris’ results in the case when C is a set of weak sharp minima for h. Received: July 24, 1998 / Accepted: November 29, 2000?Published online September 3, 2001     

Existence and multiplicity of solutions for a nonlinear Neumann problem

We consider a Neumann problem of the type -εΔu+F ^′(u(x))=0 in an open bounded subset Ω of R ⁿ , where F is a real function which has exactly k maximum points. Using Morse theory we find that, for ε suitably small, there are at least 2k nontrivial solutions of the problem and we give some qualitative information about them. Received: October 30, 1999 Published online: December 19, 2001     

Self-regular functions and new search directions for linear and semidefinite optimization

In this paper, we introduce the notion of a self-regular function. Such a function is strongly convex and smooth coercive on its domain, the positive real axis. We show that any such function induces a so-called self-regular proximity function and a corresponding search direction for primal-dual path-following interior-point methods (IPMs) for solving linear optimization (LO) problems. It is proved that the new large-update IPMs enjoy a polynomial ?(n log) iteration bound, where q≥1 is the so-called barrier degree of the kernel function underlying the algorithm. The constant hidden in the ?-symbol depends on q and the growth degree p≥1 of the kernel function. When choosing the kernel function appropriately the new large-update IPMs have a polynomial ?(lognlog) iteration bound, thus improving the currently best known bound for large-update methods by almost a factor . Our unified analysis provides also the ?(log) best known iteration bound of small-update IPMs. At each iteration, we need to solve only one linear system. An extension of the above results to semidefinite optimization (SDO) is also presented. Received: March 2000 / Accepted: December 2001?Published online April 12, 2002     

A fast algorithm for computing minimum 3-way and 4-way cuts

For an edge-weighted graph G with n vertices and m edges, we present a new deterministic algorithm for computing a minimum k-way cut for k=3,4. The algorithm runs in O(n ^k-1 F(n,m))=O(mn ^k log(n ² /m)) time and O(n ²) space for k=3,4, where F(n,m) denotes the time bound required to solve the maximum flow problem in G. The bound for k=3 matches the current best deterministic bound ?(mn ³) for weighted graphs, but improves the bound ?(mn ³) to O(n ² F(n,m))=O(min{mn ^8/3,m ^3/2 n ²}) for unweighted graphs. The bound ?(mn ⁴) for k=4 improves the previous best randomized bound ?(n ⁶) (for m=o(n ²)). The algorithm is then generalized to the problem of finding a minimum 3-way cut in a symmetric submodular system. Received: April 1999 / Accepted: February 2000?Published online August 18, 2000     

Model selection for regression on a fixed design

We deal with the problem of estimating some unknown regression function involved in a regression framework with deterministic design points. For this end, we consider some collection of finite dimensional linear spaces (models) and the least-squares estimator built on a data driven selected model among this collection. This data driven choice is performed via the minimization of some penalized model selection criterion that generalizes on Mallows' C _p . We provide non asymptotic risk bounds for the so-defined estimator from which we deduce adaptivity properties. Our results hold under mild moment conditions on the errors. The statement and the use of a new moment inequality for empirical processes is at the heart of the techniques involved in our approach. Received: 2 July 1997 / Revised version: 20 September 1999 / Published online: 6 July 2000     

Stability of solutions to parameterized nonlinear complementarity problems

n such that x≥0,  F(x,u)-v≥0 , and F(x,u)-v ^T·x=0 where these are vector inequalities. We characterize the local upper Lipschitz continuity of the (possibly set-valued) solution mapping which assigns solutions x to each parameter pair (v,u). We also characterize when this solution mapping is locally a single-valued Lipschitzian mapping (so solutions exist, are unique, and depend Lipschitz continuously on the parameters). These characterizations are automatically sufficient conditions for the more general (and usual) case where v=0. Finally, we study the differentiability properties of the solution mapping in both the single-valued and set-valued cases, in particular obtaining a new characterization of B-differentiability in the single-valued case, along with a formula for the B-derivative. Though these results cover a broad range of stability properties, they are all derived from similar fundamental principles of variational analysis. Received March 30, 1998 / Revised version received July 21, 1998 Published online January 20, 1999     

A new look at smoothing Newton methods for nonlinear complementarity problems and box constrained variational inequalities

In this paper we take a new look at smoothing Newton methods for solving the nonlinear complementarity problem (NCP) and the box constrained variational inequalities (BVI). Instead of using an infinite sequence of smoothing approximation functions, we use a single smoothing approximation function and Robinson’s normal equation to reformulate NCP and BVI as an equivalent nonsmooth equation H(u,x)=0, where H:ℜ ²ⁿ →ℜ ²ⁿ , u∈ℜ ⁿ is a parameter variable and x∈ℜ ⁿ is the original variable. The central idea of our smoothing Newton methods is that we construct a sequence {z ^k =(u ^k ,x ^k )} such that the mapping H(·) is continuously differentiable at each z ^k and may be non-differentiable at the limiting point of {z ^k }. We prove that three most often used Gabriel-Moré smoothing functions can generate strongly semismooth functions, which play a fundamental role in establishing superlinear and quadratic convergence of our new smoothing Newton methods. We do not require any function value of F or its derivative value outside the feasible region while at each step we only solve a linear system of equations and if we choose a certain smoothing function only a reduced form needs to be solved. Preliminary numerical results show that the proposed methods for particularly chosen smoothing functions are very promising. Received June 23, 1997 / Revised version received July 29, 1999?Published online December 15, 1999     

Computing the nucleolus of some combinatorially-structured games

In this paper we introduce the ℬ-prenucleolus for a transferable utility game (N,v), where ℬ⊆2 ^N . The ℬ-prenucleolus is a straightforward generalization of the ordinary prenucleolus, where only the coalitions in ℬ determine the outcome. We impose a combinatorial structure on the collection ℬ which enables us to compute the ℬ-prenucleolus in ?(n ³|ℬ|) time. The algorithm can be used for computing the nucleolus of several classes of games, among which is the class of minimum cost spanning tree games. Received: September 4, 1995 / Accepted: May 5, 1997?Published online June 8, 2000     

On semidefinite linear complementarity problems

Given a linear transformation L:? ⁿ →? ⁿ and a matrix Q∈? ⁿ , where ? ⁿ is the space of all symmetric real n×n matrices, we consider the semidefinite linear complementarity problem SDLCP(L,? ⁿ ₊,Q) over the cone ? ⁿ ₊ of symmetric n×n positive semidefinite matrices. For such problems, we introduce the P-property and its variants, Q- and GUS-properties. For a matrix A∈R ^n×n , we consider the linear transformation L _A :? ⁿ →? ⁿ defined by L _A (X):=AX+XA ^T and show that the P- and Q-properties for L _A are equivalent to A being positive stable, i.e., real parts of eigenvalues of A are positive. As a special case of this equivalence, we deduce a theorem of Lyapunov. Received: March 1999 / Accepted: November 1999?Published online April 20, 2000     

Linear preselective policies for stochastic project scheduling

In the context of stochastic resource-constrained project scheduling we introduce a novel class of scheduling policies, the linear preselective policies. They combine the benefits of preselective policies and priority policies; two classes that are well known from both deterministic and stochastic scheduling. We study several properties of this new class of policies which indicate its usefulness for computational purposes. Based on a new representation of preselective policies as and/or precedence constraints we derive efficient algorithms for computing earliest job start times and state a necessary and sufficient dominance criterion for preselective policies.  A computational experiment based on 480 instances empirically validates the theoretical findings.     

Global convergence of a class of non-interior point algorithms using Chen-Harker-Kanzow-Smale functions for nonlinear complementarity problems

We propose a class of non-interior point algorithms for solving the complementarity problems(CP): Find a nonnegative pair (x,y)∈ℝ ²ⁿ satisfying y=f(x) and x _i y _i =0 for every i∈{1,2,...,n}, where f is a continuous mapping from ℝ ⁿ to ℝ ⁿ . The algorithms are based on the Chen-Harker-Kanzow-Smale smoothing functions for the CP, and have the following features; (a) it traces a trajectory in ℝ ³ⁿ which consists of solutions of a family of systems of equations with a parameter, (b) it can be started from an arbitrary (not necessarily positive) point in ℝ ²ⁿ in contrast to most of interior-point methods, and (c) its global convergence is ensured for a class of problems including (not strongly) monotone complementarity problems having a feasible interior point. To construct the algorithms, we give a homotopy and show the existence of a trajectory leading to a solution under a relatively mild condition, and propose a class of algorithms involving suitable neighborhoods of the trajectory. We also give a sufficient condition on the neighborhoods for global convergence and two examples satisfying it. Received April 9, 1997 / Revised version received September 2, 1998? Published online May 28, 1999     

The mean value property for harmonic functions on graphs and trees

Following the Euclidean example, we introduce the strong and weak mean value property for finite variation measures on graphs. We completely characterize finite variation measures with bounded support on radial trees which have the strong mean value property. We show that for counting measures on bounded subsets of a tree with root o, the strong mean value property is equivalent to the invariance of the subset under the action of the stabilizer of o in the automorphism group. We finally characterize, using the discrete Laplacian, the finite variation measures on a generic graph which have the weak mean value property and we give a non-trivial example. Received: July 21, 2000; in final form: March 13, 2001?Published online: March 19, 2002     

Continuous modified Newton’s-type method for nonlinear operator equations

A nonlinear operator equation F(x)=0, F:H→H, in a Hilbert space is considered. Continuous Newton’s-type procedures based on a construction of a dynamical system with the trajectory starting at some initial point x ₀ and becoming asymptotically close to a solution of F(x)=0 as t→+∞ are discussed. Well-posed and ill-posed problems are investigated. Received: June 29, 2001; in final form: February 26, 2002?Published online: February 20, 2003 This paper was finished when AGR was visiting Institute for Theoretical Physics, University of Giessen. The author thanks DAAD for support     

A note on quadratic forms

We extend an interesting theorem of Yuan [12] for two quadratic forms to three matrices. Let C ₁, C ₂, C ₃ be three symmetric matrices in ℜ ^n×n , if max{x ^T C ₁ x,x ^T C ₂ x,x ^T C ₃ x}≥0 for all x∈ℜ ⁿ , it is proved that there exist t _i ≥0 (i=1,2,3) such that ∑ _i=1 ³ t _i =1 and ∑ _i=1 ³ t _i C _i has at most one negative eigenvalue. Received February 18, 1997 / Revised version received October 1, 1997? Published online June 11, 1999     

Convergence of Hybrid Steepest-Descent Methods for Generalized Variational Inequalities

In this paper, we consider the generalized variational inequality GVI（F, g, C）, where F and g are mappings from a Hilbert space into itself and C is the fixed point set of a nonexpansive mapping. We propose two iterative algorithms to find approximate solutions of the GVI（F,g, C）. Strong convergence results are established and applications to constrained generalized pseudo-inverse are included.     

On the stability of solutions to quadratic programming problems

We consider the parametric programming problem (Q _p ) of minimizing the quadratic function f(x,p):=x ^T Ax+b ^T x subject to the constraint Cx≤d, where x∈ℝ ⁿ , A∈ℝ ^n×n , b∈ℝ ⁿ , C∈ℝ ^m×n , d∈ℝ ^m , and p:=(A,b,C,d) is the parameter. Here, the matrix A is not assumed to be positive semidefinite. The set of the global minimizers and the set of the local minimizers to (Q _p ) are denoted by M(p) and M ^loc (p), respectively. It is proved that if the point-to-set mapping M ^loc (·) is lower semicontinuous at p then M ^loc (p) is a nonempty set which consists of at most ? _m,n points, where ? _m,n = is the maximal cardinality of the antichains of distinct subsets of {1,2,...,m} which have at most n elements. It is proved also that the lower semicontinuity of M(·) at p implies that M(p) is a singleton. Under some regularity assumption, these necessary conditions become the sufficient ones. Received: November 5, 1997 / Accepted: September 12, 2000?Published online November 17, 2000     

Invariant functions in Denjoy–Carleman classes

Let V be a real finite dimensional representation of a compact Lie group G. It is well known that the algebra of G-invariant polynomials on V is finitely generated, say by σ ₁, . . . , σ _p . Schwarz (Topology 14:63–68, 1975) proved that each G-invariant C ^∞-function f on V has the form f = F(σ ₁, . . . , σ _p ) for a C ^∞-function F on . We investigate this representation within the framework of Denjoy–Carleman classes. One can in general not expect that f and F lie in the same Denjoy–Carleman class C ^M (with M = (M _k )). For finite groups G and (more generally) for polar representations V, we show that for each G-invariant f of class C ^M there is an F of class C ^N such that f = F(σ ₁, . . . , σ _p ), if N is strongly regular and satisfies

Orthogonalité des caractères pour GLn sur un corps local de caractéristique non nulle

We prove the orthogonality relations for characters of GL ⁿ (F) with F a local field of positive characteristic. Using this, we establish a Jacquet-Langlands correspondence with the group of invertible elements of a central division algebra of dimension n ² over F, and the transfer of orbital integrals between these two groups. Received: 16 February 1999     

An improved rounding method and semidefinite programming relaxation for graph partition

Given an undirected graph G=(V,E) with |V|=n and an integer k between 0 and n, the maximization graph partition (MAX-GP) problem is to determine a subset S⊂V of k nodes such that an objective function w(S) is maximized. The MAX-GP problem can be formulated as a binary quadratic program and it is NP-hard. Semidefinite programming (SDP) relaxations of such quadratic programs have been used to design approximation algorithms with guaranteed performance ratios for various MAX-GP problems. Based on several earlier results, we present an improved rounding method using an SDP relaxation, and establish improved approximation ratios for several MAX-GP problems, including Dense-Subgraph, Max-Cut, Max-Not-Cut, and Max-Vertex-Cover. Received: March 10, 2000 / Accepted: July 13, 2001?Published online February 14, 2002