Entropy-like proximal algorithms based on a second-order homogeneous distance function for quasi-convex programming

We consider two classes of proximal-like algorithms for minimizing a proper lower semicontinuous quasi-convex function f(x) subject to non-negative constraints . The algorithms are based on an entropy-like second-order homogeneous distance function. Under the assumption that the global minimizer set is nonempty and bounded, we prove the full convergence of the sequence generated by the algorithms, and furthermore, obtain two important convergence results through imposing certain conditions on the proximal parameters. One is that the sequence generated will converge to a stationary point if the proximal parameters are bounded and the problem is continuously differentiable, and the other is that the sequence generated will converge to a solution of the problem if the proximal parameters approach to zero. Numerical experiments are done for a class of quasi-convex optimization problems where the function f(x) is a composition of a quadratic convex function from to and a continuously differentiable increasing function from to , and computational results indicate that these algorithms are very promising in finding a global optimal solution to these quasi-convex problems. Shaohua Pan work was partially supported by the Doctoral Starting-up Foundation (05300161) of GuangDong Province. Member of Mathematics Division, National Center for Theoretical Sciences, Taipei Office. Jein-Shan Chen work is partially supported by National Science Council of Taiwan.     

Computing a Celis-Dennis-Tapia trust-region step for equality constrained optimization

We study an approach for minimizing a convex quadratic function subject to two quadratic constraints. This problem stems from computing a trust-region step for an SQP algorithm proposed by Celis, Dennis and Tapia (1985) for equality constrained optimization. Our approach is to reformulate the problem into a univariate nonlinear equation()=0 where the function() is continuous, at least piecewise differentiable and monotone. Well-established methods then can be readily applied. We also consider an extension of our approach to a class of non-convex quadratic functions and show that our approach is applicable to reduced Hessian SQP algorithms. Numerical results are presented indicating that our algorithm is reliable, robust and has the potential to be used as a building block to construct trust-region algorithms for small-sized problems in constrained optimization.This research was performed while the author was on a postdoctoral appointment in the Department of Mathematical Sciences, Rice University, Houston, TX, USA and was supported in part by AFOSR 85-0243 and DOE DEFG05-86ER 25017.     

Towards equitable distribution via proportional equity constraints

We propose a new approach to the formulation of models for solving problems of equitable distribution. Equity is achieved by the use of proportional equity constraints which require that each recipient must receive ashare of the total distribution which falls within a prespecified range. The algorithmic implications of this new approach are illustrated using two mathematical models. We consider the problem of maximizing total flow in a multiterminal network with proportional equity constraints. An efficient algorithm for solving this problem is provided. As a further application, we show that the introduction of such constraints still permits easy solutions for the linear knapsack problem.     

A (2 + )-Approximation Scheme for Minimum Domination on Circle Graphs

The main result of this paper is a (2 + )-approximation scheme for the minimum dominating set problem on circle graphs. We first present an O(n²) time 8-approximation algorithm for this problem and then extend it to an time (2 + )-approximation scheme for this problem. Here n and m are the number of vertices and the number of edges of the circle graph. We then present simple modifications to this algorithm that yield (3 + )-approximation schemes for the minimum connected and the minimum total dominating set problems on circle graphs. Keil (1993, Discrete Appl. Math.42, 51–63) shows that these problems are NP-complete for circle graphs and leaves open the problem of devising approximation algorithms for them. These are the first O(1)-approximation algorithms for domination problems on circle graphs.     

Cutting Plane Algorithms for Nonlinear Semi-Definite Programming Problems with Applications

We will propose an outer-approximation (cutting plane) method for minimizing a function f X subject to semi-definite constraints on the variables XR ⁿ. A number of efficient algorithms have been proposed when the objective function is linear. However, there are very few practical algorithms when the objective function is nonlinear. An algorithm to be proposed here is a kind of outer-approximation(cutting plane) method, which has been successfully applied to several low rank global optimization problems including generalized convex multiplicative programming problems and generalized linear fractional programming problems, etc. We will show that this algorithm works well when f is convex and n is relatively small. Also, we will provide the proof of its convergence under various technical assumptions.     

Projection methods for non-linear programming

Several algorithms are presented for solving the non-linear programming problem, based on variable-metric projections of the gradient of the objective function into a local approximation to the constraints. The algorithms differ in the nature of this approximation. Inequality constraints are dealt with by selecting at each step a subset of active constraints to treat as equalities, this subset being the smallest necessary to ensure that the new point remains feasible. Some numerical results are given for the Colville problems.Paper presented at the 7^th Mathematical Programming Symposium, The Hague, September 1970.     

Stronger Lagrangian bounds by use of slack variables: Applications to machine scheduling problems

Lagrangian relaxation is a powerful bounding technique that has been applied successfully to manyNP-hard combinatorial optimization problems. The basic idea is to see anNP-hard problem as an easy-to-solve problem complicated by a number of nasty side constraints. We show that reformulating nasty inequality constraints as equalities by using slack variables leads to stronger lower bounds. The trick is widely applicable, but we focus on a broad class of machine scheduling problems for which it is particularly useful. We provide promising computational results for three problems belonging to this class for which Lagrangian bounds have appeared in the literature: the single-machine problem of minimizing total weighted completion time subject to precedence constraints, the two-machine flow-shop problem of minimizing total completion time, and the single-machine problem of minimizing total weighted tardiness.     

Reflection-Projection Method for Convex Feasibility Problems with an Obtuse Cone

The convex feasibility problem asks to find a point in the intersection of finitely many closed convex sets in Euclidean space. This problem is of fundamental importance in the mathematical and physical sciences, and it can be solved algorithmically by the classical method of cyclic projections.In this paper, the case where one of the constraints is an obtuse cone is considered. Because the nonnegative orthant as well as the set of positive-semidefinite symmetric matrices form obtuse cones, we cover a large and substantial class of feasibility problems. Motivated by numerical experiments, the method of reflection-projection is proposed: it modifies the method of cyclic projections in that it replaces the projection onto the obtuse cone by the corresponding reflection.This new method is not covered by the standard frameworks of projection algorithms because of the reflection. The main result states that the method does converge to a solution whenever the underlying convex feasibility problem is consistent. As prototypical applications, we discuss in detail the implementation of two-set feasibility problems aiming to find a nonnegative [resp. positive semidefinite] solution to linear constraints in ⁿ [resp. in , the space of symmetric n×n matrices] and we report on numerical experiments. The behavior of the method for two inconsistent constraints is analyzed as well.     

Global optimization and simulated annealing

In this paper we are concerned with global optimization, which can be defined as the problem of finding points on a bounded subset of ⁿ in which some real valued functionf assumes its optimal (maximal or minimal) value.We present a stochastic approach which is based on the simulated annealing algorithm. The approach closely follows the formulation of the simulated annealing algorithm as originally given for discrete optimization problems. The mathematical formulation is extended to continuous optimization problems, and we prove asymptotic convergence to the set of global optima. Furthermore, we discuss an implementation of the algorithm and compare its performance with other well-known algorithms. The performance evaluation is carried out for a standard set of test functions from the literature.     

A randomized scheme for speeding up algorithms for linear and convex programming problems with high constraints-to-variables ratio

We extend Clarkson's randomized algorithm for linear programming to a general scheme for solving convex optimization problems. The scheme can be used to speed up existing algorithms on problems which have many more constraints than variables. In particular, we give a randomized algorithm for solving convex quadratic and linear programs, which uses that scheme together with a variant of Karmarkar's interior point method. For problems withn constraints,d variables, and input lengthL, ifn = (d ²), the expected total number of major Karmarkar's iterations is O(d ²(logn)L), compared to the best known deterministic bound of O( L). We also present several other results which follow from the general scheme.     

A little theorem of the bigℳ in interior point algorithms

When we apply interior point algorithms to various problems including linear programs, convex quadratic programs, convex programs and complementarity problems, we often embed an original problem to be solved in an artificial problem having a known interior feasible solution from which we start the algorithm. The artificial problem involves a constant (or constants) which we need to choose large enough to ensure the equivalence between the artificial problem and the original problem. Theoretically, we can always assign a positive number of the order O(2 ^L ) to in linear cases, whereL denotes the input size of the problem. Practically, however, such a large number is impossible to implement on computers. If we choose too large, we may have numerical instability and/or computational inefficiency, while the artificial problem with not large enough will never lead to any solution of the original problem. To solve this difficulty, this paper presents a little theorem of the big, which will enable us to find whether is not large enough, and to update during the iterations of the algorithm even if we start with a smaller. Applications of the theorem are given to a polynomial-time potential reduction algorithm for positive semi-definite linear complementarity problems, and to an artificial self-dual linear program which has a close relation with the primal—dual interior point algorithm using Lustig's limiting feasible direction vector.     

A theoretical approximation scheme for Stackelberg problems

We consider the Stackelberg problem corresponding to a two-player game in which one of the two players has the leadership in playing the game. We present a general approach for approximating the considered hierarchical programming problem by a sequence of two-level optimization problems. From a practical point of view, we also give some results for asymptotically Stackelberg approximating sequences and for problems with perturbed constraints.This paper is based upon results first presented at Journées Fermat: Mathematics for Optimization, Toulouse, France, May 1985.     

A Zero-Space algorithm for Negative Cost Cycle Detection in networks

This paper is concerned with the problem of checking whether a network with positive and negative costs on its arcs contains a negative cost cycle. The Negative Cost Cycle Detection (NCCD) problem is one of the more fundamental problems in network design and finds applications in a number of domains ranging from Network Optimization and Operations Research to Constraint Programming and System Verification. As per the literature, approaches to this problem have been either Relaxation-based or Contraction-based. We introduce a fundamentally new approach for negative cost cycle detection; our approach, which we term as the Stressing Algorithm, is based on exploiting the connections between the NCCD problem and the problem of checking whether a system of difference constraints is feasible. The Stressing Algorithm is an incremental, comparison-based procedure which is as efficient as the fastest known comparison-based algorithm for this problem. In particular, on a network with n vertices and m edges, the Stressing Algorithm takes O(mn) time to detect the presence of a negative cost cycle or to report that none exists. A very important feature of the Stressing Algorithm is that it uses zero extra space; this is in marked contrast to all known algorithms that require Ω(n) extra space. It is well known that the NCCD problem is closely related to the Single Source Shortest Paths (SSSP) problem, i.e., the problem of determining the shortest path distances of all the vertices in a network, from a specified source; indeed most algorithms in the literature for the NCCD problem are modifications of approaches to the SSSP problem. At this juncture, it is not clear whether the Stressing Algorithm could be extended to solve the SSSP problem, even if O(n) extra space is available.     

A continuation method for (strongly) monotone variational inequalities

We consider the variational inequality problem, denoted by VIP(X, F), whereF is a strongly monotone function and the convex setX is described by some inequality (and possibly equality) constraints. This problem is solved by a continuation (or interior-point) method, which solves a sequence of certain perturbed variational inequality problems. These perturbed problems depend on a parameter > 0. It is shown that the perturbed problems have a unique solution for all values of > 0, and that any sequence generated by the continuation method converges to the unique solution of VIP(X,F) under a well-known linear independence constraint qualification (LICQ). We also discuss the extension of the continuation method to monotone variational inequalities and present some numerical results obtained with a suitable implementation of this method. © 1998 The Mathematical Programming Society, Inc. Published by Elsevier Science B.V.     

Implicit Convex Polygons

Convex polygons in the plane can be defined explicitly as an ordered list of vertices, or given implicitly, for example by a list of linear constraints. The latter representation has been considered in several fields such as facility location, robotics and computer graphics. In this paper, we investigate many fundamental geometric problems for implicitly represented polygons and give simple and fast algorithms that are easy to implement. We uncover an interesting partition of the problems into two classes: those that exhibit an (nlogn) lower bound on their complexity, and those that yield O(n) time algorithms via prune-and-search methods.     

Asymptotically Optimal Algorithms for Job Shop Scheduling and Packet Routing

We propose asymptotically optimal algorithms for the job shop scheduling and packet routing problems. We propose a fluid relaxation for the job shop scheduling problem in which we replace discrete jobs with the flow of a continuous fluid. We compute an optimal solution of the fluid relaxation in closed form, obtain a lower bound C_max to the job shop scheduling problem, and construct a feasible schedule from the fluid relaxation with objective value at most where the constant in the O( · ) notation is independent of the number of jobs, but it depends on the processing time of the jobs, thus producing an asymptotically optimal schedule as the total number of jobs tends to infinity. If the initially present jobs increase proportionally, then our algorithm produces a schedule with value at most C_max + O(1). For the packet routing problem with fixed paths the previous algorithm applies directly. For the general packet routing problem we propose a linear programming relaxation that provides a lower bound C_max and an asymptotically optimal algorithm that uses the optimal solution of the relaxation with objective value at most Unlike asymptotically optimal algorithms that rely on probabilistic assumptions, our proposed algorithms make no probabilistic assumptions and they are asymptotically optimal for all instances with a large number of jobs (packets). In computational experiments our algorithms produce schedules which are within 1% of optimality even for moderately sized problems.     

An exact penalty function algorithm for optimal control problems with control and terminal equality constraints,part 2

In Part 1 of this paper, implementable and conceptual versions of an algorithm for optimal control problems with control constraints and terminal equality constraints were presented. It was shown that anyL accumulation points of control sequences generated by the algorithms satisfy necessary conditions of optimality. Since such accumulation points need not exist, it is shown in this paper that control sequences generated by the algorithms always have accumulation points in the sense of control measure, and these accumulation points satisfy optimality conditions for the corresponding relaxed control problem.This work was supported by the United Kingdom Science Research Council, by the US Army Research Office, Contract No. DAA-29-73-C-0025, and by the National Science Foundation, Grant No. ENG-73-08214-A01.     

Uncertain convex programs: randomized solutions and confidence levels

Many engineering problems can be cast as optimization problems subject to convex constraints that are parameterized by an uncertainty or instance parameter. Two main approaches are generally available to tackle constrained optimization problems in presence of uncertainty: robust optimization and chance-constrained optimization. Robust optimization is a deterministic paradigm where one seeks a solution which simultaneously satisfies all possible constraint instances. In chance-constrained optimization a probability distribution is instead assumed on the uncertain parameters, and the constraints are enforced up to a pre-specified level of probability. Unfortunately however, both approaches lead to computationally intractable problem formulations.In this paper, we consider an alternative randomized or scenario approach for dealing with uncertainty in optimization, based on constraint sampling. In particular, we study the constrained optimization problem resulting by taking into account only a finite set of N constraints, chosen at random among the possible constraint instances of the uncertain problem. We show that the resulting randomized solution fails to satisfy only a small portion of the original constraints, provided that a sufficient number of samples is drawn. Our key result is to provide an efficient and explicit bound on the measure (probability or volume) of the original constraints that are possibly violated by the randomized solution. This volume rapidly decreases to zero as N is increased.This work is supported in part by the European Commission under the project HYBRIDGE IST-2001-32460, and by MIUR under the project New methods for Identification and Adaptive Control for Industrial Systems, and the FIRB project Learning, randomization and guaranteed predictive inference for complex uncertain systems.Acknowledgement We wish to thank Professor Arkadi Nemirovski for his encouragement in pursuing this line of research. We also acknowledge the many valuable comments from anonymous reviewers that helped improve this paper.     

Cost Based Filtering for the Constrained Knapsack Problem

We present cost based filtering methods for Knapsack Problems (KPs). Cost based filtering aims at fixing variables with respect to the objective function. It is an important technique when solving complex problems such as Quadratic Knapsack Problems, or KPs with additional constraints (Constrained Knapsack Problems (CKPs)). They evolve, e.g., when Constraint Based Column Generation is applied to appropriate optimization problems. We develop new reduction algorithms for KP. They are used as propagation routines for the CKP with (nlogn) preprocessing time and (n) time per call. This sums up to an amortized time (n) for (logn) incremental calls where the subsequent problems may differ with respect to arbitrary sets of necessarily included and excluded items.     

A Class of Global Optimization Problems as Models of the Phase Unwrapping Problem

Let I={(i,j) i=1, 2,..., N ₁, j=1, 2,..., N ₂} and let U=U_i,j, (i,j)I be a discrete real function defined on I. Let []₂ be modulus 2, we define W:I , ) as follows W=[U]₂. The function U will be called phase function and the function W will be called wrapped phase function. The phase unwrapping problem consists in recovering U from some knowledge of W. This problem is not well defined, that is infinitely many functions U correspond to the same function W, and must be `regularized' to be satisfactorily solvable. We propose several formulations of the phase unwrapping problem as an integer nonlinear minimum cost flow problem on a network. Numerical algorithms to solve the minimum cost flow problems obtained are proposed. The phase unwrapping problem is the key problem in interferometry, we restrict our attention to the SAR (Synthetic Aperture Radar) interferometry problem. We compare the different formulations of the phase unwrapping problem proposed starting from the analysis of the numerical experience obtained with the numerical algorithms proposed on synthetic and real SAR interferometry data. The real data are taken from the ERS missions of the European Space Agency (ESA).