Randomized algorithms for the separation of point sets and for solving quadratic programs

A randomized algorithm for finding a hyperplane separating two finite point sets in the Euclidean space ^d and a randomized algorithm for solving linearly constrained general convex quadratic problems are proposed. The expected running time of the separating algorithm isO(dd! (m + n)), wherem andn are cardinalities of sets to be separated. The expected running time of the algorithm for solving quadratic problems isO(dd! s) wheres is the number of inequality constraints. These algorithms are based on the ideas of Seidel's linear programming algorithm [6]. They are closely related to algorithms of [8], [2], and [9] and belong to an abstract class of algorithms investigated in [1]. The algorithm for solving quadratic problems has some features of the one proposed in [7].This research was done when the author was supported by the Alexander von Humboldt Foundation, Germany.On leave from the Institute of Mathematics and Mechanics, Ural Department of the Russian Academy of Sciences, 620219 Ekaterinburg, S. Kovalevskaya str. 16, Russia.     

Linearly convergent descent methods for the unconstrained minimization of convex quadratic splines

We propose two linearly convergent descent methods for finding a minimizer of a convex quadratic spline and establish global error estimates for the iterates. One application of such descent methods is to solve convex quadratic programs, since they can be reformulated as problems of unconstrained minimization of convex quadratic splines. In particular, we derive several new linearly convergent algorthms for solving convex quadratic programs. These algorithms could be classified as row-action methods, matrix-splitting methods, and Newton-type methods.     

An Exterior Newton Method for Strictly Convex Quadratic Programming

We propose an exterior Newton method for strictly convex quadratic programming (QP) problems. This method is based on a dual formulation: a sequence of points is generated which monotonically decreases the dual objective function. We show that the generated sequence converges globally and quadratically to the solution (if the QP is feasible and certain nondegeneracy assumptions are satisfied). Measures for detecting infeasibility are provided. The major computation in each iteration is to solve a KKT-like system. Therefore, given an effective symmetric sparse linear solver, the proposed method is suitable for large sparse problems. Preliminary numerical results are reported.     

Dual support method for solving convex quadratic programs

In this article, we present a new dual method for solving convex (but not strictly convex) quadratic programs (QPs). Our method is the generalization of the dual support method, developed by Gabasov and co-workers in 1981, for solving convex QPs. It proceeds in two phases: the first is to construct the initial support, called coordinator support, for the problem and the second is to achieve the optimality of the problem. Results of numerical experiments are given comparing our approach with the active-set method.     

An interior point method for quadratic programs based on conjugate projected gradients

We propose an interior point method for large-scale convex quadratic programming where no assumptions are made about the sparsity structure of the quadratic coefficient matrixQ. The interior point method we describe is a doubly iterative algorithm that invokes aconjugate projected gradient procedure to obtain the search direction. The effect is thatQ appears in a conjugate direction routine rather than in a matrix factorization. By doing this, the matrices to be factored have the same nonzero structure as those in linear programming. Further, one variant of this method istheoretically convergent with onlyone matrix factorization throughout the procedure.     

Some properties of generalized proximal point methods for quadratic and linear programming

The proximal point method for convex optimization has been extended recently through the use of generalized distances (e.g., Bregman distances) instead of the Euclidean one. One advantage of these extensions is the possibility of eliminating certain constraints (mainly positivity) from the subproblems, transforming an inequality constrained problem into a sequence of unconstrained or equality constrained problems. We consider here methods obtained using a certain class of Bregman functions applied to convex quadratic (including linear) programming, which are of the interior-point type. We prove that the limit of the sequence generated by the method lies in the relative interior of the solution set, and furthermore is the closest optimal solution to the initial point, in the sense of the Bregman distance. These results do not hold for the standard proximal point method, i.e., when the square of the Euclidean norm is used as the Bregman distance.The research leading to this paper was partially supported by CNPq Grant 301280/86.We thank two anonymous referees whose comments and suggestions allowed us to improve our results in a significant way.     

Generalized proximal point algorithm for convex optimization

Ha (Ref. 1) recently introduced a generalized proximal point algorithm for solving a generalized equation. In this note, we present a generalized proximal point algorithm for convex optimization problems based on Ha's work. The idea behind this algorithm is that, instead of adding a quadratic term to all the variables, we add a quadratic term to a subset of the variables. We extend the criteria for approximate solutions given by Rockafellar (Ref. 2) and Auslender (Ref. 3) and present convergence results. Finally, we show how this algorithm can be applied to solve block-angular linear and quadratic programming problems.     

On the solution set of convex problems and its numerical application

The solution set of a convex problem is enough interested. In this paper we have a discussion to determine it. The conditions and proofs in this note essentially differs from the previous well-known works. We also have a new result determining the solution of minimum norm for quadratic problems.     

Generalized derivatives of distance functions and the existence of nearest points

The relationships between the generalized directional derivative of the distance function and the existence of nearest points as well as some geometry properties in Banach spaces are studied. It is proved in the present paper that the condition that for each closed subset G$G$ of X$X$ and x∈X?G$x\in X?G$, the Clarke, Michel-Penot, Dini or modified Dini directional derivative of the distance function is 1 or −1 implying the existence of the nearest points to x$x$ from G$G$ is equivalent to X$X$ being compactly locally uniformly convex. Similar results for uniqueness of the nearest point are also established.     

Interior path following primal-dual algorithms. part II: Convex quadratic programming

We describe a primal-dual interior point algorithm for convex quadratic programming problems which requires a total of number of iterations, whereL is the input size. Each iteration updates a penalty parameter and finds an approximate Newton direction associated with the Karush-Kuhn-Tucker system of equations which characterizes a solution of the logarithmic barrier function problem. The algorithm is based on the path following idea. The total number of arithmetic operations is shown to be of the order of O(n ³ L).     

An interior point method for general large-scale quadratic programming problems

In this paper, we present an interior point algorithm for solving both convex and nonconvex quadratic programs. The method, which is an extension of our interior point work on linear programming problems efficiently solves a wide class of largescale problems and forms the basis for a sequential quadratic programming (SQP) solver for general large scale nonlinear programs. The key to the algorithm is a three-dimensional cost improvement subproblem, which is solved at every interation. We have developed an approximate recentering procedure and a novel, adaptive big-M Phase I procedure that are essential to the sucess of the algorithm. We describe the basic method along with the recentering and big-M Phase I procedures. Details of the implementation and computational results are also presented.Contribution of the National Institute of Standards and Tedchnology and not subject to copyright in the United States. This research was supported in part by ONR Contract N-0014-87-F0053.     

An interior-point method for multifractional programs with convex constraints

We present an interior-point method for a family of multi-fractional programs with convex constraints. The programs under consideration consist of minimizing the maximum of a finite number of linear fractions over some convex set. First, we present a simple shortstep algorithm for solving such multifractional programs, and we show that, under suitable assumptions, the convergence of the short-step algorithm is weakly polynomial in a sense specified below. Then, we describe a practical implementation of the proposed method, and we report results of numerical experiments with this algorithm. These results suggest that the proposed method is a viable alternative to the standard Dinkelbach-type algorithms for solving multifractional programs.The authors would like to thank Professor A. S. Nemirovsky for stimulating discussions via electronic mail. We are grateful to two anonymous referees for comments and suggestions that improved our paper.     

A randomized scheme for speeding up algorithms for linear and convex programming problems with high constraints-to-variables ratio

We extend Clarkson's randomized algorithm for linear programming to a general scheme for solving convex optimization problems. The scheme can be used to speed up existing algorithms on problems which have many more constraints than variables. In particular, we give a randomized algorithm for solving convex quadratic and linear programs, which uses that scheme together with a variant of Karmarkar's interior point method. For problems withn constraints,d variables, and input lengthL, ifn = (d ²), the expected total number of major Karmarkar's iterations is O(d ²(logn)L), compared to the best known deterministic bound of O( L). We also present several other results which follow from the general scheme.     

A conjugate gradient method for the unconstrained minimization of strictly convex quadratic splines

In this paper, we show that an analogue of the classical conjugate gradient method converges linearly when applied to solving the problem of unconstrained minimization of a strictly convex quadratic spline. Since a strictly convex quadratic program with simple bound constraints can be reformulated as unconstrained minimization of a strictly convex quadratic spline, the conjugate gradient method is used to solve the unconstrained reformulation and find the solution of the original quadratic program. In particular, if the solution of the original quadratic program is nondegenerate, then the conjugate gradient method finds the solution in a finite number of iterations. This author's research is partially supported by the NASA/Langley Research Center under grant NCC-1-68 Supplement-15.     

Accelerated Landweber iteration with convex penalty for linear inverse problems in Banach spaces

In recent years, Landweber iteration has been extended to solve linear inverse problems in Banach spaces by incorporating non-smooth convex penalty functionals to capture features of solutions. This method is known to be slowly convergent. However, because it is simple to implement, it still receives a lot of attention. By making use of the subspace optimization technique, we propose an accelerated version of Landweber iteration with non-smooth convex penalty which significantly speeds up the method. Numerical simulations are given to test the efficiency.     

On certain optimization methods with finite-step inner algorithms for convex finite-dimensional problems with inequality constraints

Numerical methods are proposed for solving finite-dimensional convex problems with inequality constraints satisfying the Slater condition. A method based on solving the dual to the original regularized problem is proposed and justified for problems having a strictly uniformly convex sum of the objective function and the constraint functions. Conditions for the convergence of this method are derived, and convergence rate estimates are obtained for convergence with respect to the functional, convergence with respect to the argument to the set of optimizers, and convergence to the g-normal solution. For more general convex finite-dimensional minimization problems with inequality constraints, two methods with finite-step inner algorithms are proposed. The methods are based on the projected gradient and conditional gradient algorithms. The paper is focused on finite-dimensional problems obtained by approximating infinite-dimensional problems, in particular, optimal control problems for systems with lumped or distributed parameters.     

On a proximal point method for convex optimization in banach spaces

We analyze the behavior of a parallel proximal point method for solving convex optimization problems in reflexive Banach spaces. Similar algorithms were known to converge under the implicit assumption that the norm of the space is Hilbertian. We extend the area of applicability of the proximal point method to solving convex optimization problems in Banach spaces on which totally convex functions can be found. This includes the class of all smooth uniformly convex Banach spaces. Also, our convergence results leave more flexibility for the choice of the penalty function involved in the algorithm and, in this way, allow simplification of the computational procedure.     

An implementation of a parallel primal-dual interior point method for block-structured linear programs

An implementation of the primal-dual predictor-corrector interior point method is specialized to solve block-structured linear programs with side constraints. The block structure of the constraint matrix is exploited via parallel computation. The side constraints require the Cholesky factorization of a dense matrix, where a method that exploits parallelism for the dense Cholesky factorization is used. For testing, multicommodity flow problems were used. The resulting implementation is 65%–90% efficient, depending on the problem instance. For a problem with K commodities, an approximate speedup for the interior point method of 0.8K is realized.     

MATHEMATICAL ANALYSIS OF SOME NEURAL NETWORKS FOR SOLVING LINEAR AND QUADRATIC PROGRAMMING

ＺＨＡＮＧＸＩＡＮＧＳＵＮ（章祥荪）（ＩｎｓｔｉｔｕｔｅｏｆＡｐｐｌｉｅｄＭａｔｈｅｍａｔｉｃｓｔｈｅＣｈｉｎｅｓｅＡｃａｄｅｍｙｏｆＳｃｉｅｎｃｅｓ，Ｂｅｉｊｉｎｇ１０００８０，Ｃｈｉｎａ）ＲｅｃｅｉｖｅｄＪｕｎｅ１８，１９９４．Ｔｈｉｓｗｏｒｋｉ...     

A recursive algorithm for finding the minimum norm point in a polytope and a pair of closest points in two polytopes

For a given pair of finite point setsP andQ in some Euclidean space we consider two problems: Problem 1 of finding the minimum Euclidean norm point in the convex hull ofP and Problem 2 of finding a minimum Euclidean distance pair of points in the convex hulls ofP andQ. We propose a finite recursive algorithm for these problems. The algorithm is not based on the simplicial decomposition of convex sets and does not require to solve systems of linear equations.