A dual exact penalty formulation for the linear complementarity problem

We consider a dual exact penalty formulation for the monotone linear complementarity problem. Tihonov regularization is then used to reduce the solution of the problem to the solution of a sequence of positive-definite, symmetric quadratic programs. A modified form of an SOR method due to Mangasarian is proposed to solve these quadratic programs. We also indicate how to obtain approximate solutions to predefined tolerance by solving a single quadratic program, in special cases.This research was sponsored by US Army Contract DAAG29-80-C-0041, by National Science Foundation Grants DCR-8420963 and MCS-8102684, and AFSOR Grant AFSOR-ISSA-85-0880.     

A finite algorithm for the least two-norm solution of a linear program 1

By perturbing properly a linear program to a separable quadratic program it is possible to solve the latter in its dual variable space by iterative techniques such as sparsity-preserving SOR (successive overtaxation techniques). In this way large sparse linear programs can be handled.

In this paper we give a new computational criterion to check whether the solution of the perturbed quadratic program provides the least 2-norm solution of the original linear program. This criterion improves on the criterion proposed in an earlier paper.

We also describe an algorithm for solving linear programs which is based on the SOR methods. The main property of this algorithm is that, under mild assumptions, it finds the least 2-norm solution of a linear program in a finite number of iteration.s     

Block SOR methods for rank-deficient least-squares problems

Many papers have discussed preconditioned block iterative methods for solving full rank least-squares problems. However very few papers studied iterative methods for solving rank-deficient least-squares problems. Miller and Neumann (1987) proposed the 4-block SOR method for solving the rank-deficient problem. Here a 2-block SOR method and a 3-block SOR method are proposed to solve such problem. The convergence of the block SOR methods is studied. The optimal parameters are determined. Comparison between the 2-block SOR method and the 3-block SOR method is given also.     

线性块SOR二级迭代法的收敛性质

内迭代次数充分大时,求解非奇异线性方程组的块SOR二级迭代法与经典的块SOR方法有相同的收敛性和大致相等的收敛速度.因此,用于块SOR方法有效的松弛因子,同样可有效地用于块SOR二级迭代法.     

Preconditioned sor methods for generalized least-squares problems

1.IntroductionThegeneralizedleastsquaresproblem,definedasmin(Ax--b)"W--'(Ax--b),(1.1)xacwhereAERm",m>n,bERm,andWERm'misasymmetricandpositivedefinitematrix,isfrequentlyfoundwhensolvingproblemsinstatistics,engineeringandeconomics.Forexample,wegetgeneralizedleastsquaresproblemswhensolvingnonlinearregressionanalysisbyquasi-likelihoodestimation,imagereconstructionproblemsandeconomicmodelsobtainedbythemaximumlikelihoodmethod(of.[1,21).Paige[3,4]investigatestheproblemexplicitly.Hechangestheorig…     

Initialization in semidefinite programming via a self-dual skew-symmetric embedding

The formulation of interior point algorithms for semidefinite programming has become an active research area, following the success of the methods for large-scale linear programming. Many interior point methods for linear programming have now been extended to the more general semidefinite case, but the initialization problem remained unsolved.In this paper we show that the initialization strategy of embedding the problem in a self-dual skew-symmetric problem can also be extended to the semidefinite case. This method also provides a solution for the initialization of quadratic programs and it is applicable to more general convex problems with conic formulation.     

Some remarks on completely vectorizing point Gauss-Seidel while using the natural ordering

A common statement in papers in the vectorization field is to note that point SOR methods with the natural ordering cannot be vectorized. The usual approach is to reorder the unknowns using a red-black or diagonal ordering and vectorize that. In this paper we construct a point Gauss-Seidel iteration which completely vectorizes and still uses the natural ordering. The work here also applies to both point SOR and single program, multiple data (SPMD) parallel computer architectures. When this approach is reasonable to use is also shown.     

A compact formulation of an elastoplastic analysis problem

Two important problems in the area of engineering plasticity are limit load analysis and elastoplastic analysis. It is well known that these two problems can be formulated as linear and quadratic programming problems, respectively (Refs. 1–2). In applications, the number of variables in each of these mathematical programming problems tends to be large. Consequently, it is important to have efficient numerical methods for their solution. The purpose of this paper is to present a method which allows the quadratic programming formulation of the elastoplastic analysis to be reformulated as an equivalent quadratic programming problem which has significantly fewer variables than the original formulation. Indeed, in Section 4, we will present details of an example for which the original quadratic programming formulation required 297 variables and for which the equivalent formulation presented here required only two variables. The method is based on a characterization of the entire family of optimal solutions for a linear programming problem.This research was supported by the Natural Science and Engineering Council of Canada under Grant No. A8189 and by a Leave Fellowship from the Social Sciences and Humanities Research Council of Canada. The author takes pleasure in acknowledging many stimulating discussions with Professor D. E. Grierson.     

Extrapolated Gauss-Seidel I and SOR methods for least-squares problems

Summary Recently, special attention has been given in the literature, to the problems of accurately computing the least-squares solution of very largescale over-determined systems of linear equations which occur in geodetic applications. In particular, it has been suggested that one can solve such problems iteratively by applying the block SOR (Successive Overrelaxation) and EGS1 (Extrapolated Gauss Seidel 1) plus semi-iterative methods to a linear system with coefficient matrix 2-cyclic or 3-cyclic. The comparison of 2-block SOR and 3-block SOR was made in [1] and showed that the 2-block SOR is better. In [6], the authors also proved that 3-block EGS1-SI is better than 3-block SOR. Here, we first show that the 2-block DJ (Double Jacobi)-SI, GS-SI and EGS1-SI methods are equivalent and all of them are equivalent to the 3-block EGS1-SI method; then, we prove that the composite methods and 2-block SOR have the same asymptotic rate of convergence, but the former has a better average rate of convergence; finally, numerical experiments are reported, and confirm that the 3-block EGS1-SI is better than the 2-block SOR.     

Pairwise reactive sor algorithm for quadratic programming of net import spatial equilibrium models

An iterative method based on the successive overrelaxation (SOR) is proposed to solve quadratic programming of net important spatial equilibrium models. The algorithm solves the problem by updating the variables pairwise at each iteration. The principal feature of the algorithm is that the lagrange multipliers corresponding to the constraints do not have to be calculated at each iteration as is the case in SOR based algorithms. Yet the Lagrange multipliers can easily be extracted from the solution values.This research was partially supported by grants from the James F. Kember Foundation and the School of Business, Loyola University of Chicago.     

Convergence of a direct-iterative method for large-scale least-squares problems

In 1975 Chen and Gentleman suggested a 3-block SOR method for solving least-squares problems, based on a partitioning scheme for the observation matrix A into

$\text{A=}\text{A}{}_1\text{A}{}_2$

where A₁ is square and nonsingular. In many cases A₁ is obvious from the nature of the problem. This combined direct-iterative method was discussed further and applied to angle adjustment problems in geodesy, where A₁ is easily formed and is large and sparse, by Plemmons in 1979. Recently, Niethammer, de Pillis, and Varga have rekindled interest in this method by correcting and extending the SOR convergence interval. The purpose of our paper is to discuss an alternative formulation of the problem leading to a 2-block SOR method. For this formulation it is shown that the resulting direct-iterative method always converges for sufficiently small SOR parameter, in contrast to the 3-block formulation. Formulas for the optimum SOR parameter and the resulting asymptotic convergence factor, based upon 6A₂A^-1₁6₂, are given. Furthermore, it is shown that this 2-cyclic block SOR method always gives better convergence results than the 3-cyclic one for the same amount of work per iteration. The direct part of the algorithm requires only a sparse-matrix factorization of A₁. Our purpose here is to establish theoretical convergence results, in line with the purpose of the recent paper by Niethammer, de Pillis, and Varga. Practical considerations of choosing A₁ in certain applications and of estimating the resulting 6A₂A^-1₁6₂ will be addressed elsewhere.     

A simple and effective algorithm for the MaxMin diversity problem

The challenge of maximizing the diversity of a collection of points arises in a variety of settings, including the setting of search methods for hard optimization problems. One version of this problem, called the Maximum Diversity Problem (MDP), produces a quadratic binary optimization problem subject to a cardinality constraint, and has been the subject of numerous studies. This study is focused on the Maximum Minimum Diversity Problem (MMDP) but we also introduce a new formulation using MDP as a secondary objective. We propose a fast local search based on separate add and drop operations and on simple tabu mechanisms. Compared to previous local search approaches, the complexity of searching for the best move at each iteration is reduced from quadratic to linear; only certain streamlining calculations might (rarely) require quadratic time per iteration. Furthermore, the strong tabu rules of the drop strategy ensure a powerful diversification capacity. Despite its simplicity, the approach proves superior to most of the more advanced methods from the literature, yielding optimally-proved solutions for many problems in a matter of seconds and even attaining a new lower bound.     

Duality-based algorithms for total-variation-regularized image restoration

Image restoration models based on total variation (TV) have become popular since their introduction by Rudin, Osher, and Fatemi (ROF) in 1992. The dual formulation of this model has a quadratic objective with separable constraints, making projections onto the feasible set easy to compute. This paper proposes application of gradient projection (GP) algorithms to the dual formulation. We test variants of GP with different step length selection and line search strategies, including techniques based on the Barzilai-Borwein method. Global convergence can in some cases be proved by appealing to existing theory. We also propose a sequential quadratic programming (SQP) approach that takes account of the curvature of the boundary of the dual feasible set. Computational experiments show that the proposed approaches perform well in a wide range of applications and that some are significantly faster than previously proposed methods, particularly when only modest accuracy in the solution is required.     

A nonconvex quadratic optimization approach to the maximum edge weight clique problem

The maximum edge weight clique (MEWC) problem, defined on a simple edge-weighted graph, is to find a subset of vertices inducing a complete subgraph with the maximum total sum of edge weights. We propose a quadratic optimization formulation for the MEWC problem and study characteristics of this formulation in terms of local and global optimality. We establish the correspondence between local maxima of the proposed formulation and maximal cliques of the underlying graph, implying that the characteristic vector of a MEWC in the graph is a global optimizer of the continuous problem. In addition, we present an exact algorithm to solve the MEWC problem. The algorithm is a combinatorial branch-and-bound procedure that takes advantage of a new upper bound as well as an efficient construction heuristic based on the proposed quadratic formulation. Results of computational experiments on some benchmark instances are also presented.     

Experiments in numerical methods for a problem in combustion modeling

In this paper, we have shown that the numerical method of lines can be used effectively to solve time dependent combustion models in one spatial dimension. By the numerical method of lines (NMOL), we mean the reduction of a system of partial differential equations to a system of ordinary differential equations (ODE's), followed by the solution of this ODE system with an appropriate ODE solver. We used finite differences for the spatial discretization and a variant of the GEAR package for the ODE's.We have presented various solution methods of interest for the nonlinear algebraic system in this setting; that is, in the corrector iteration section of the GEAR package applied to combustion models. These methods include Newton/block SOR (SOR denotes successive over-relaxation), block SOR/Newton, Newton/block-diagonal Jacobian, Newton/kinetics-only Jacobian, and Newton/block symmetric SOR. These methods have in common their lack of frequent use in ODE software and their eady applicability to partial differential equations in more than one spatial dimension.Finally, we have given the results of numerical tests, run on the CDC-7600 and Cray-1 computers. By so doing, we indicate the more promising nonlinear system solvers for the NMOL solution of combustion models.     

A new result in the theory and computation of the least-norm solution of a linear program

By perturbing properly a linear program to a separable quadratic program, it is possible to solve the latter in its dual variable space by iterative techniques such as sparsity-preserving SOR (successive overrelaxation) algorithms. The main result of this paper gives an effective computational criterion to check whether the solutions of the perturbed quadratic programs provide the least-norm solution of the original linear program.This research was sponsored by the United States Army under Contract No. DAAG29-80-C-0041. This material is based upon work supported by the National Science Foundation, Grant Nos. DCR-84-20963 and DMS-82-109050, and by the Italian National Research Council (CNR).The author wishes to thank Professor O. L. Mangasarian for his helpful comments which helped to improve the paper.     

A dual differentiable exact penalty function

A new penalty function is associated with an inequality constrained nonlinear programming problem via its dual. This penalty function is globally differentiable if the functions defining the original problem are twice globally differentiable. In addition, the penalty parameter remains finite. This approach reduces the original problem to a simple problem of maximizing a globally differentiable function on the product space of a Euclidean space and the nonnegative orthant of another Euclidean space. Many efficient algorithms exist for solving this problem. For the case of quadratic programming, the penalty function problem can be solved effectively by successive overrelaxation (SOR) methods which can handle huge problems while preserving sparsity features. Sponsored by the United States Army under Contract No. DAAG 29-80-C-0041. This material is based upon work supported by the National Science Foundation under Grants No. MCS-790166 and ENG-7903881.     

Optimal multi-period mean–variance policy under no-shorting constraint

We consider in this paper the mean–variance formulation in multi-period portfolio selection under no-shorting constraint. Recognizing the structure of a piecewise quadratic value function, we prove that the optimal portfolio policy is piecewise linear with respect to the current wealth level, and derive the semi-analytical expression of the piecewise quadratic value function. One prominent feature of our findings is the identification of a deterministic time-varying threshold for the wealth process and its implications for market settings. We also generalize our results in the mean–variance formulation to utility maximization with no-shorting constraint.     

Integer programming models and linearizations for the traveling car renter problem

The traveling car renter problem (CaRS) is an extension of the classical traveling salesman problem (TSP) where different cars are available for use during the salesman’s tour. In this study we present three integer programming formulations for CaRS, of which two have quadratic objective functions and the other has quadratic constraints. The first model with a quadratic objective function is grounded on the TSP interpreted as a special case of the quadratic assignment problem in which the assignment variables refer to visitation orders. The second model with a quadratic objective function is based on the Gavish and Grave’s formulation for the TSP. The model with quadratic constraints is based on the Dantzig–Fulkerson–Johnson’s formulation for the TSP. The formulations are linearized and implemented in two solvers. An experiment with 50 instances is reported.     

On the supports of functions

In this paper, finite difference and finite element methods are used with nonlinear SOR to solve the problems of minimizing strict convex functionals. The functionals are discretized by both methods and some numerical quadrature formula. The convergence of such discretization is guaranteed and will be discussed. As for the convergence of the iterative process, it is necessary to vary the relaxation parameter in each iterations. In addition, for the model catenoid problem, boundary grid refinements play an essential role in the proposed nonlinear SOR algorithm. Numerical results which illustrate the importance of the grid refinements will be presented.