Comparison of a special-purpose algorithm with general-purpose algorithms for solving geometric programming problems

We study the performance of four general-purpose nonlinear programming algorithms and one special-purpose geometric programming algorithm when used to solve geometric programming problems. Experiments are reported which show that the special-purpose algorithm GGP often finds approximate solutions more quickly than the general-purpose algorithm GRG2, but is usually not significantly more efficient than GRG2 when greater accuracy is required. However, for some of the most difficult test problems attempted, GGP was dramatically superior to all of the other algorithms. The other algorithms are usually not as efficient as GGP or GRG2. The ellipsoid algorithm is most robust.This work was supported in part by the National Science Foundation, Grant No. MCS-81-02141.     

A quasi-Newton trust-region method

The classical trust-region method for unconstrained minimization can be augmented with a line search that finds a point that satisfies the Wolfe conditions. One can use this new method to define an algorithm that simultaneously satisfies the quasi-Newton condition at each iteration and maintains a positive-definite approximation to the Hessian of the objective function. This new algorithm has strong global convergence properties and is robust and efficient in practice.     

A trust-region strategy for minimization on arbitrary domains

We present a trust-region method for minimizing a general differentiable function restricted to an arbitrary closed set. We prove a global convergence theorem. The trust-region method defines difficult subproblems that are solvable in some particular cases. We analyze in detail the case where the domain is a Euclidean ball. For this case we present numerical experiments where we consider different Hessian approximations.Work partially supported by FAPESP (Grants 90-3724-6 and 91-2441-3), FINEP, CNPq and FAEP-UNICAMP.     

An RQP algorithm using a differentiable exact penalty function for inequality constrained problems

In this paper we propose a recursive quadratic programming algorithm for nonlinear programming problems with inequality constraints that uses as merit function a differentiable exact penalty function. The algorithm incorporates an automatic adjustment rule for the selection of the penalty parameter and makes use of an Armijo-type line search procedure that avoids the need to evaluate second order derivatives of the problem functions. We prove that the algorithm possesses global and superlinear convergence properties. Numerical results are reported.     

Recursive quadratic programming algorithm that uses an exact augmented Lagrangian function

An algorithm for nonlinear programming problems with equality constraints is presented which is globally and superlinearly convergent. The algorithm employs a recursive quadratic programming scheme to obtain a search direction and uses a differentiable exact augmented Lagrangian as line search function to determine the steplength along this direction. It incorporates an automatic adjustment rule for the selection of the penalty parameter and avoids the need to evaluate second-order derivatives of the problem functions. Some numerical results are reported.     

An analysis of reduced Hessian methods for constrained optimization

We study the convergence properties of reduced Hessian successive quadratic programming for equality constrained optimization. The method uses a backtracking line search, and updates an approximation to the reduced Hessian of the Lagrangian by means of the BFGS formula. Two merit functions are considered for the line search: the ₁ function and the Fletcher exact penalty function. We give conditions under which local and superlinear convergence is obtained, and also prove a global convergence result. The analysis allows the initial reduced Hessian approximation to be any positive definite matrix, and does not assume that the iterates converge, or that the matrices are bounded. The effects of a second order correction step, a watchdog procedure and of the choice of null space basis are considered. This work can be seen as an extension to reduced Hessian methods of the well known results of Powell (1976) for unconstrained optimization.This author was supported, in part, by National Science Foundation grant CCR-8702403, Air Force Office of Scientific Research grant AFOSR-85-0251, and Army Research Office contract DAAL03-88-K-0086.This author was supported by the Applied Mathematical Sciences subprogram of the Office of Energy Research, U.S. Department of Energy, under contracts W-31-109-Eng-38 and DE FG02-87ER25047, and by National Science Foundation Grant No. DCR-86-02071.     

Global convergence without the assumption of linear independence for a trust-region algorithm for constrained optimization

A trust-region algorithm for solving the equality constrained optimization problem is presented. This algorithm uses the Byrd and Omojokun way of computing the trial steps, but it differs from the Byrd and Omojokun algorithm in the way steps are evaluated. A global convergence theory for this new algorithm is presented. The main feature of this theory is that the linear independence assumption on the gradients of the constraints is not assumed.This research was supported in part by the Center for Research on Parallel Computation, by Grant NSF-CCR-91-20008, and by the REDI Foundation.     

Numerical solution of a nonlinear parabolic control problem by a reduced SQP method

We consider a control problem for a nonlinear diffusion equation with boundary input that occurs when heating ceramic products in a kiln. We interpret this control problem as a constrained optimization problem, and we develop a reduced SQP method that presents for this problem a new and efficient approach of its numerical solution. As opposed to Newton's method for the unconstrained problem, where at each iteration the state must be computed from a set of nonlinear equations,in the proposed algorithm only the linearized state equations need to be solved. Furthermore, by use of a secant update formula, the calculation of exact second derivatives is avoided. In this way the algorithm achieves a substantial decrease in the total cost compared to the implementation of Newton's method in [2]. Our method is practicable with regard to storage requirements, and by choosing an appropriate representation for the null space of the Jacobian of the constraints we are able to exploit the sparsity pattern of the Jacobian in the course of the iteration. We conclude with a presentation of numerical examples that demonstrate the fast two-step superlinear convergence behavior of the method.     

Globally and superlinearly convergent trust-region algorithm for convex SC1-minimization problems and its application to stochastic programs

A function mapping from ⁿ to is called an SC¹-function if it is differentiable and its derivative is semismooth. A convex SC¹-minimization problem is a convex minimization problem with an SC¹-objective function and linear constraints. Applications of such minimization problems include stochastic quadratic programming and minimax problems. In this paper, we present a globally and superlinearly convergent trust-region algorithm for solving such a problem. Numerical examples are given on the application of this algorithm to stochastic quadratic programs.This work was supported by the Australian Research Council.We are indebted to Dr. Xiaojun Chen for help in the computation. We are grateful to two anonymous referees for their comments and suggestions, which improved the presentation of this paper.     

Computational experience with Davidon's least-square algorithm

Computational results are presented for Davidon's new least-square algorithm. Computational experience with this algorithm is reported which motivated the development of a production code version of the algorithm. Several heuristic modifications, which have been added, are described. Fifteen zero-residual test problems have been used in comparing the new production code version with two established versions of the Levenberg-Marquardt algorithm. The production code version of Davidon's least-square algorithm performed faster and used less function evaluations than the Levenberg-Marquardt algorithm in almost every case of the test problems.It is a pleasure to acknowledge and thank M. Thomas, R. & I. Consultant, Western Illinois University Computer Center, for writing the timing routine and taking the time to run the comparison tests on the IBM 360/50. Part of this work was also performed at the Applied Mathematics Division of Argonne National Laboratory under the auspices of the US Energy Research and Development Administration.     

A Piecewise Line-Search Technique for Maintaining the Positive Definitenes of the Matrices in the SQP Method

A technique for maintaining the positive definiteness of the matrices in the quasi-Newton version of the SQP algorithm is proposed. In our algorithm, matrices approximating the Hessian of the augmented Lagrangian are updated. The positive definiteness of these matrices in the space tangent to the constraint manifold is ensured by a so-called piecewise line-search technique, while their positive definiteness in a complementary subspace is obtained by setting the augmentation parameter. In our experiment, the combination of these two ideas leads to a new algorithm that turns out to be more robust and often improves the results obtained with other approaches.     

等式约束优化一个新的SQP算法

提出了一个处理等式约束优化问题新的SQP算法,该算法通过求解一个增广Lagrange函数的拟Newton方法推导出一个等式约束二次规划子问题,从而获得下降方向．罚因子具有自动调节性,并能避免趋于无穷．为克服Maratos效应采用增广Lagrange函数作为效益函数并结合二阶步校正方法．在适当的条件下,证明算法是全局收敛的,并且具有超线性收敛速度．     

A sparse nonlinear optimization algorithm

One of the most effective numerical techniques for solving nonlinear programming problems is the sequential quadratic programming approach. Many large nonlinear programming problems arise naturally in data fitting and when discretization techniques are applied to systems described by ordinary or partial differential equations. Problems of this type are characterized by matrices which are large and sparse. This paper describes a nonlinear programming algorithm which exploits the matrix sparsity produced by these applications. Numerical experience is reported for a collection of trajectory optimization problems with nonlinear equality and inequality constraints.The authors wish to acknowledge the insightful contributions of Dr. William Huffman.     

Numerical experiments with variations of the Gauss-Newton algorithm for nonlinear least squares

In this paper, the classical Gauss-Newton method for the unconstrained least squares problem is modified by introducing a quasi-Newton approximation to the second-order term of the Hessian. Various quasi-Newton formulas are considered, and numerical experiments show that most of them are more efficient on large residual problems than the Gauss-Newton method and a general purpose minimization algorithm based upon the BFGS formula. A particular quasi-Newton formula is shown numerically to be superior. Further improvements are obtained by using a line search that exploits the special form of the function.     

An algorithm of sequential systems of linear equations for nonlinear optimization problems with arbitrary initial point

For current sequential quadratic programming (SQP) type algorithms, there exist two problems: (i) in order to obtain a search direction, one must solve one or more quadratic programming subproblems per iteration, and the computation amount of this algorithm is very large. So they are not suitable for the large-scale problems; (ii) the SQP algorithms require that the related quadratic programming subproblems be solvable per iteration, but it is difficult to be satisfied. By using ε-active set procedure with a special penalty function as the merit function, a new algorithm of sequential systems of linear equations for general nonlinear optimization problems with arbitrary initial point is presented. This new algorithm only needs to solve three systems of linear equations having the same coefficient matrix per iteration, and has global convergence and local superlinear convergence. To some extent, the new algorithm can overcome the shortcomings of the SQP algorithms mentioned above. Project partly supported by the National Natural Science Foundation of China and Tianyuan Foundation of China.     

A sparse sequential quadratic programming algorithm

Described here is the structure and theory for a sequential quadratic programming algorithm for solving sparse nonlinear optimization problems. Also provided are the details of a computer implementation of the algorithm along with test results. The algorithm maintains a sparse approximation to the Cholesky factor of the Hessian of the Lagrangian. The solution to the quadratic program generated at each step is obtained by solving a dual quadratic program using a projected conjugate gradient algorithm. An updating procedure is employed that does not destroy sparsity.     

非线性约束最优化一族超线性收敛的可行方法

本文建立求解非线性不等式约束最优化一族含参数的可行方法．算法每次迭代仅需解一个规模较小的二次规划．在一定的假设条件下,证明了算法族的全局收敛性和超线性收敛性．     

Computational experience using an edge search algorithm for linear reverse convex programs

This paper presents computational experience with a rather straight forward implementation of an edge search algorithm for obtaining the globally optimal solution for linear programs with an additional reverse convex constraint. The paper's purpose is to provide a collection of problems, with known optimal solutions, and performance information for an edge search implementation so that researchers may have some benchmarks with which to compare new methods for reverse convex programs or concave minimization problems. There appears to be nothing in the literature that provides computational experience with a basic edge search procedure. The edge search implementation uses a depth first strategy. As such, this paper's implementation of the edge search algorithm is a modification of Hillestad's algorithm [11]. A variety of test problems is generated by using a modification of the method of Sung and Rosen [20], as well as a new method that is presented in this paper. Test problems presented may be obtained at ftp://newton.ee.ucla.edu/nonconvex/pub/.     

Robust Recursive Quadratic Programming Algorithm Model with Global and Superlinear Convergence Properties

A new, robust recursive quadratic programming algorithm model based on a continuously differentiable merit function is introduced. The algorithm is globally and superlinearly convergent, uses automatic rules for choosing the penalty parameter, and can efficiently cope with the possible inconsistency of the quadratic search subproblem. The properties of the algorithm are studied under weak a priori assumptions; in particular, the superlinear convergence rate is established without requiring strict complementarity. The behavior of the algorithm is also investigated in the case where not all of the assumptions are met. The focus of the paper is on theoretical issues; nevertheless, the analysis carried out and the solutions proposed pave the way to new and more robust RQP codes than those presently available.     

A SUCCESSIVE QUADRATIC PROGRAMMING ALGORITHM FOR SDP RELAXATION OF MAX-BISECTION

A successive quadratic programming algorithm for solving SDP relaxation of Max- Bisection is provided and its convergence result is given.The step-size in the algorithm is obtained by solving n easy quadratic equations without using the linear search technique.The numerical experiments show that this algorithm is rather faster than the interior-point method.