Some Randomized Algorithms for Convex Quadratic Programming

We adapt some randomized algorithms of Clarkson [3] for linear programming to the framework of so-called LP-type problems, which was introduced by Sharir and Welzl [10]. This framework is quite general and allows a unified and elegant presentation and analysis. We also show that LP-type problems include minimization of a convex quadratic function subject to convex quadratic constraints as a special case, for which the algorithms can be implemented efficiently, if only linear constraints are present. We show that the expected running times depend only linearly on the number of constraints, and illustrate this by some numerical results. Even though the framework of LP-type problems may appear rather abstract at first, application of the methods considered in this paper to a given problem of that type is easy and efficient. Moreover, our proofs are in fact rather simple, since many technical details of more explicit problem representations are handled in a uniform manner by our approach. In particular, we do not assume boundedness of the feasible set as required in related methods. Accepted 7 May 1997     

A method of linearizations for linearly constrained nonconvex nonsmooth minimization

A readily implementable algorithm is given for minimizing a (possibly nondifferentiable and nonconvex) locally Lipschitz continuous functionf subject to linear constraints. At each iteration a polyhedral approximation tof is constructed from a few previously computed subgradients and an aggregate subgradient, which accumulates the past subgradient information. This aproximation and the linear constraints generate constraints in the search direction finding subproblem that is a quadratic programming problem. Then a stepsize is found by an approximate line search. All the algorithm's accumulation points are stationary. Moreover, the algorithm converges whenf happens to be convex.     

Randomized search directions in descent methods for minimizing certain quasidifferentiable functions

Several descent methods have recently been proposed for minimizing smooth compositions of max-type functions. The methods generate many search directions at each iteration. This paper shows that a random choice of only two search directions at each iteration suffices for retaining convergence to inf-stationary points with probability 1. This technique may decrease significantly the work in quadratic programming and line searches, thus enabling efficient implementations of the methods.     

A modified Weiszfeld algorithm for the Fermat-Weber location problem

This paper gives a new, simple, monotonically convergent, algorithm for the Fermat-Weber location problem, with extensions covering more general cost functions. Received: September 1999 / Accepted: January 2001?Published online April 12, 2001     

A General Approach to Convergence Properties of Some Methods for Nonsmooth Convex Optimization

Based on the notion of the ε -subgradient, we present a unified technique to establish convergence properties of several methods for nonsmooth convex minimization problems. Starting from the technical results, we obtain the global convergence of: (i) the variable metric proximal methods presented by Bonnans, Gilbert, Lemaréchal, and Sagastizábal, (ii) some algorithms proposed by Correa and Lemaréchal, and (iii) the proximal point algorithm given by Rockafellar. In particular, we prove that the Rockafellar—Todd phenomenon does not occur for each of the above mentioned methods. Moreover, we explore the convergence rate of {||x _k || } and {f(x _k ) } when {x _k } is unbounded and {f(x _k ) } is bounded for the non\-smooth minimization methods (i), (ii), and (iii). Accepted 15 October 1996     

Finding Mixed Cells in the Mixed Volume Computation

In practice, finding mixed cells in certain polyhedral subdivisions plays a dominating role when a polyhedral homotopy is employed to approximate all isolated zeros of polynomial systems. This paper gives a new algorithm for the mixed cell computation via a new formulation of the underlying linear programming problems. Numerical results show that the algorithm provides a major advance in the speed of computation with much less memory requirements. March 17, 2000. Final version received: November 2, 2000. Online publication: February 20, 2001.     

Unicity Results for General Linear Semi-Infinite Optimization Problems Using a New Concept of Active Constraints

We consider parametric semi-infinite optimization problems without the usual asssumptions on the continuity of the involved mappings and on the compactness of the index set counting the inequalities. We establish a characterization of those optimization problems which have a unique or strongly unique solution and which are stable under small pertubations. This result generalizes a well-known theorem of Nürnberger. The crucial roles in our investigations are a new concept of active constraints, a generalized Slater's condition, and a Kuhn—Tucker-type theorem. Finally, we give some applications in vector optimization, for approximation problems in normed spaces, and in the stability of the minimal value. Accepted 5 August 1996     

A modified version of Tuy's method for solving d.c. programing problem

We present a modified version of Tuy's method for solving d.c. programming problems described in canonical form and some results of computational experiments.     

Partially finite convex programming,Part I: Quasi relative interiors and duality theory

We study convex programs that involve the minimization of a convex function over a convex subset of a topological vector space, subject to a finite number of linear inequalities. We develop the notion of the quasi relative interior of a convex set, an extension of the relative interior in finite dimensions. We use this idea in a constraint qualification for a fundamental Fenchel duality result, and then deduce duality results for these problems despite the almost invariable failure of the standard Slater condition. Part II of this work studies applications to more concrete models, whose dual problems are often finite-dimensional and computationally tractable.     

Partially finite convex programming,Part II: Explicit lattice models

In Part I of this work we derived a duality theorem for partially finite convex programs, problems for which the standard Slater condition fails almost invariably. Our result depended on a constraint qualification involving the notion ofquasi relative interior. The derivation of the primal solution from a dual solution depended on the differentiability of the dual objective function: the differentiability of various convex functions in lattices was considered at the end of Part I. In Part II we shall apply our results to a number of more concrete problems, including variants of semi-infinite linear programming,L ¹ approximation, constrained approximation and interpolation, spectral estimation, semi-infinite transportation problems and the generalized market area problem of Lowe and Hurter (1976). As in Part I, we shall use lattice notation extensively, but, as we illustrated there, in concrete examples lattice-theoretic ideas can be avoided, if preferred, by direct calculation.     

Solution of a General Linear Complementarity Problem Using Smooth Optimization and Its Application to Bilinear Programming and LCP

This paper addresses a General Linear Complementarity Problem (GLCP) that has found applications in global optimization. It is shown that a solution of the GLCP can be computed by finding a stationary point of a differentiable function over a set defined by simple bounds on the variables. The application of this result to the solution of bilinear programs and LCPs is discussed. Some computational evidence of its usefulness is included in the last part of the paper. Accepted 28 June 1999. Online publication 4 December 2000.     

A Nonmonotone Trust Region Method for Nonlinear Programming with Simple Bound Constraints

In this paper we propose a nonmonotone trust region algorithm for optimization with simple bound constraints. Under mild conditions, we prove the global convergence of the algorithm. For the monotone case it is also proved that the correct active set can be identified in a finite number of iterations if the strict complementarity slackness condition holds, and so the proposed algorithm reduces finally to an unconstrained minimization method in a finite number of iterations, allowing a fast asymptotic rate of convergence. Numerical experiments show that the method is efficient. Accepted 5 September 2000. Online publication 4 December 2000.     

Quadratic Convergence of Newton's Method for Convex Interpolation and Smoothing

   Abstract. In this paper, we prove that Newton's method for convex best interpolation is locally quadratically convergent, giving an answer to a question of Irvine, Marin, and Smith [7] and strengthening a result of Andersson and Elfving [1] and our previous work [5]. A damped Newton-type method is presented which has global quadratic convergence. Analogous results are obtained for the convex smoothing problem. Numerical examples are presented.     

A family of variable metric proximal methods

We consider conceptual optimization methods combining two ideas: the Moreau—Yosida regularization in convex analysis, and quasi-Newton approximations of smooth functions. We outline several approaches based on this combination, and establish their global convergence. Then we study theoretically the local convergence properties of one of these approaches, which uses quasi-Newton updates of the objective function itself. Also, we obtain a globally and superlinearly convergent BFGS proximal method. At each step of our study, we single out the assumptions that are useful to derive the result concerned.     

A Note on Mascarenhas' Counterexample about Global Convergence of the Affine Scaling Algorithm

Mascarenhas gave an instance of linear programming problems to show that the long-step affine scaling algorithm can fail to converge to an optimal solution with the step-size λ=0.999 . In this note, we give a simple and clear geometrical explanation for this phenomenon in terms of the Newton barrier flow induced by projecting the homogeneous affine scaling vector field conically onto a hyperplane where the objective function is constant. Based on this interpretation, we show that the algorithm can fail for "any" λ greater than about 0.91 (a more precise value is 0.91071), which is considerably shorter than λ = 0.95 and 0.99 recommended for efficient implementations. Accepted 17 February 1998     

A Relaxation of a Min-Cut Problem in an Anisotropic Continuous Network

Strang [18] introduced optimization problems on a Euclidean domain which are closely related with problems in mechanics and noted that the problems are regarded as continuous versions of famous max-flow and min-cut problems. In [15] we generalized the problems and called the generalized problems max-flow and min-cut problems of Strang's type. In this paper we formulate a relaxed version of the min-cut problem of Strang's type and prove the existence of optimal solutions under some suitable conditions. The conditions are essential. In fact, there is an example of the relaxed version which has no optimal solutions if the conditions are not fulfilled. We give such an example in the final section. Accepted 8 October 1998     

Convergence Properties of Projection and Contraction Methods for Variational Inequality Problems

In this paper we develop the convergence theory of a general class of projection and contraction algorithms (PC method), where an extended stepsize rule is used, for solving variational inequality (VI) problems. It is shown that, by defining a scaled projection residue, the PC method forces the sequence of the residues to zero. It is also shown that, by defining a projected function, the PC method forces the sequence of projected functions to zero. A consequence of this result is that if the PC method converges to a nondegenerate solution of the VI problem, then after a finite number of iterations, the optimal face is identified. Finally, we study local convergence behavior of the extragradient algorithm for solving the KKT system of the inequality constrained VI problem. \keywords{Variational inequality, Projection and contraction method, Predictor-corrector stepsize, Convergence property.} \amsclass{90C30, 90C33, 65K05.} Accepted 5 September 2000. Online publication 16 January 2001.     

A Dynamical System Associated with Newton's Method for Parametric Approximations of Convex Minimization Problems

We study the existence and asymptotic convergence when t→+∞ for the trajectories generated by where is a parametric family of convex functions which approximates a given convex function f we want to minimize, and ε(t) is a parametrization such that ε(t)→ 0 when t→+∞ . This method is obtained from the following variational characterization of Newton's method: where H is a real Hilbert space. We find conditions on the approximating family and the parametrization to ensure the norm convergence of the solution trajectories u(t) toward a particular minimizer of f . The asymptotic estimates obtained allow us to study the rate of convergence as well. The results are illustrated through some applications to barrier and penalty methods for linear programming, and to viscosity methods for an abstract noncoercive variational problem. Comparisons with the steepest descent method are also provided. Accepted 5 December 1996     

Adaptive Mollifiers for High Resolution Recovery of Piecewise Smooth Data from its Spectral Information

We discuss the reconstruction of piecewise smooth data from its (pseudo-) spectral information. Spectral projections enjoy superior resolution provided the data is globally smooth, while the presence of jump discontinuities is responsible for spurious O (1) Gibbs oscillations in the neighborhood of edges and an overall deterioration of the unacceptable first-order convergence in rate. The purpose is to regain the superior accuracy in the piecewise smooth case, and this is achieved by mollification. Here we utilize a modified version of the two-parameter family of spectral mollifiers introduced by Gottlieb and Tadmor [GoTa85]. The ubiquitous one-parameter, finite-order mollifiers are based on dilation . In contrast, our mollifiers achieve their high resolution by an intricate process of high-order cancellation . To this end, we first implement a localization step using an edge detection procedure [GeTa00a, b]. The accurate recovery of piecewise smooth data is then carried out in the direction of smoothness away from the edges, and adaptivity is responsible for the high resolution. The resulting adaptive mollifier greatly accelerates the convergence rate, recovering piecewise analytic data within exponential accuracy while removing the spurious oscillations that remained in [GoTa85]. Thus, these adaptive mollifiers offer a robust, general-purpose ``black box' procedure for accurate post-processing of piecewise smooth data. March 29, 2001. Final version received: August 31, 2001.