Acceleration of the leastpth algorithm for minimax optimization with engineering applications

Over the past few years a number of researchers in mathematical programming and engineering became very interested in both the theoretical and practical applications of minimax optimization. The purpose of the present paper is to present a new method of solving the minimax optimization problem and at the same time to apply it to nonlinear programming and to three practical engineering problems. The original problem is defined as a modified leastpth objective function which under certain conditions has the same optimum as the original problem. The advantages of the present approach over the Bandler-Charalambous leastpth approach are similar to the advantages of the augmented Lagrangians approach for nonlinear programming over the standard penalty methods.This work was supported by the National Research Council of Canada under Grant A4414, and from the University of Waterloo.     

Nonlinear leastpth optimization and nonlinear programming

Over the past few years a number of researchers in mathematical programming became very interested in the method of the Augmented Lagrangian to solve the nonlinear programming problem. The main reason being that the Augmented Lagrangian approach overcomes the ill-conditioning problem and the slow convergence of the penalty methods. The purpose of this paper is to present a new method of solving the nonlinear programming problem, which has similar characteristics to the Augmented Lagrangian method. The original nonlinear programming problem is transformed into the minimization of a leastpth objective function which under certain conditions has the same optimum as the original problem. Convergence and rate of convergence of the new method is also proved. Furthermore numerical results are presented which illustrate the usefulness of the new approach to nonlinear programming.This work was supported by the National Research Council of Canada and by the Department of Combinatorics and Optimization of the University of Waterloo.     

On the need for special purpose algorithms for minimax eigenvalue problems

It has been recently reported that minimax eigenvalue problems can be formulated as nonlinear optimization problems involving smooth objective and constraint functions. This result seems very appealing since minimax eigenvalue problems are known to be typically nondifferentiable. In this paper, we show, however, that general purpose nonlinear optimization algorithms usually fail to find a solution to these smooth problems even in the simple case of minimization of the maximum eigenvalue of an affine family of symmetric matrices, a convex problem for which efficient algorithms are available.This work was supported in part by NSF Engineering Research Centers Program No. NSFD-CDR-88-03012 and NSF Grant DMC-84-20740. The author wishes to thank Drs. M. K. H. Fan and A. L. Tits for their many useful suggestions.     

Convergence of one-leg methods for nonlinear neutral delay integro-differential equations

Some convergence results of one-leg methods for nonlinear neutral delay integro-differential equations (NDIDEs) are obtained. It is proved that a one-leg method is E (or EB) -convergent of order p for nonlinear NDIDEs if and only if it is A-stable and consistent of order p in classical sense for ODEs, where p = 1, 2. A numerical example that confirms the theoretical results is given in the end of this paper. This work was supported by National Natural Science Foundation of China (Grant No. 10871164), the Natural Science Foundation of Hunan Province (Grant No. 08JJ6002), and the Scientific Research Fund of Changsha University of Science and Technology (Grant No. 1004259)     

Parameter Estimation with the Augmented Lagrangian Method for a Parabolic Equation

In this paper, we investigate the numerical identification of the diffusion parameters in a linear parabolic problem. The identification is formulated as a constrained minimization problem. By using the augmented Lagrangian method, the inverse problem is reduced to a coupled nonlinear algebraic system, which can be solved efficiently with the preconditioned conjugate gradient method. Finally, we present some numerical experiments to show the efficiency of the proposed methods, even for identifying highly discontinuous parameters.This work was partially supported by the Research Council of Norway, Grant NFR-128224/431.     

On solving three classes of nonlinear programming problems via simple differentiable penalty functions

We consider the following classes of nonlinear programming problems: the minimization of smooth functions subject to general constraints and simple bounds on the variables; the nonlinearl ₁-problem; and the minimax problem. Numerically reliable methods for solving problems in each of these classes, based upon exploiting the structure of the problem in constructing simple differentiable penalty functions, are presented.This research was made possible by NSERC Grant No. A8442.The author would like to thank Mrs. J. Selwood of the Department of Combinatories and Optimization, University of Waterloo, Ontario, Canada for her excellent typesetting.This work was carried out in the Department of Combinatories and Optimization, University of Waterloo, Waterloo, Ontario, Canada.     

An outer approximation method for minimizing the product of several convex functions on a convex set

This paper addresses the minimization of the product ofp convex functions on a convex set. It is shown that this nonconvex problem can be converted to a concave minimization problem withp variables, whose objective function value is determined by solving a convex minimization problem. An outer approximation method is proposed for obtaining a global minimum of the resulting problem. Computational experiments indicate that this algorithm is reasonable efficient whenp is less than 4.This research was partly supported by Grant-in-Aid for Scientific Research of the Ministry of Education, Science and Culture, Grant No. (C)03832018 and (C)04832010.     

On conditions for optimality in least pth approximation withp → ∞

This paper presents a theoretical discussion of the necessary and sufficient conditions for optimality in generalized nonlinear leastpth approximation problems forp . In the limit, the conditions for a minimax approximation are derived, as is to be expected. Numerical examples involving the modeling of a linear time-invariant fourth-order system by a second-order model and the design of quarter-wave transmission-line transformers illustrate the results.This work was supported by the National Research Council of Canada under Grant No. A7239 and by a Frederick Gardner Cottrell Grant from the Research Corporation. This paper was presented at the 9th Annual Allerton Conference on Circuit and System Theory, Urbana, Illinois, October 6–8, 1971. The authors thank Mrs. J. R. Popovi for helping to correct Example 4.1.     

Ordinary convex programs without a duality gap

In the Kuhn-Tucker theory of nonlinear programming, there is a close relationship between the optimal solutions to a given minimization problem and the saddlepoints of the corresponding Lagrangian function. It is shown here that, if the constraint functions and objective function arefaithfully convex in a certain broad sense and the problem has feasible solutions, then theinf sup andsup inf of the Lagrangian are necessarily equal.This work was supported in part by the Air Force Office of Scientific Research under Grant No. AF-AFOSR-1202-67B.     

Variable Programming: A Generalized Minimax Problem. Part II: Algorithms

In this part of the two-part series of papers, algorithms for solving some variable programming (VP) problems proposed in Part I are investigated. It is demonstrated that the non-differentiability and the discontinuity of the maximum objective function, as well as the summation objective function in the VP problems constitute difficulty in finding their solutions. Based on the principle of statistical mechanics, we derive smooth functions to approximate these non-smooth objective functions with specific activated feasible sets. By transforming the minimax problem and the corresponding variable programming problems into their smooth versions we can solve the resulting problems by some efficient algorithms for smooth functions. Relevant theoretical underpinnings about the smoothing techniques are established. The algorithms, in which the minimization of the smooth functions is carried out by the standard quasi-Newton method with BFGS formula, are tested on some standard minimax and variable programming problems. The numerical results show that the smoothing techniques yield accurate optimal solutions and that the algorithms proposed are feasible and efficient.This work was supported by the RGC grant CUHK 152/96H of the Hong Kong Research Grant Council.     

Numerical Comparison of Augmented Lagrangian Algorithms for Nonconvex Problems

Augmented Lagrangian algorithms are very popular tools for solving nonlinear programming problems. At each outer iteration of these methods a simpler optimization problem is solved, for which efficient algorithms can be used, especially when the problems are large. The most famous Augmented Lagrangian algorithm for minimization with inequality constraints is known as Powell-Hestenes-Rockafellar (PHR) method. The main drawback of PHR is that the objective function of the subproblems is not twice continuously differentiable. This is the main motivation for the introduction of many alternative Augmented Lagrangian methods. Most of them have interesting interpretations as proximal point methods for solving the dual problem, when the original nonlinear programming problem is convex. In this paper a numerical comparison between many of these methods is performed using all the suitable problems of the CUTE collection.This author was supported by ProNEx MCT/CNPq/FAPERJ 171.164/2003, FAPESP (Grants 2001/04597-4 and 2002/00094-0 and 2003/09169-6) and CNPq (Grant 302266/2002-0).This author was partially supported by CNPq-Brasil and CDCHT-Venezuela.This author was supported by ProNEx MCT/CNPq/FAPERJ 171.164/2003, FAPESP (Grant 2001/04597-4) and CNPq.     

Nonlinear complementarity as unconstrained and constrained minimization

The nonlinear complementarity problem is cast as an unconstrained minimization problem that is obtained from an augmented Lagrangian formulation. The dimensionality of the unconstrained problem is the same as that of the original problem, and the penalty parameter need only be greater than one. Another feature of the unconstrained problem is that it has global minima of zero at precisely all the solution points of the complementarity problem without any monotonicity assumption. If the mapping of the complementarity problem is differentiable, then so is the objective of the unconstrained problem, and its gradient vanishes at all solution points of the complementarity problem. Under assumptions of nondegeneracy and linear independence of gradients of active constraints at a complementarity problem solution, the corresponding global unconstrained minimum point is locally unique. A Wolfe dual to a standard constrained optimization problem associated with the nonlinear complementarity problem is also formulated under a monotonicity and differentiability assumption. Most of the standard duality results are established even though the underlying constrained optimization problem may be nonconvex. Preliminary numerical tests on two small nonmonotone problems from the published literature converged to degenerate or nondegenerate solutions from all attempted starting points in 7 to 28 steps of a BFGS quasi-Newton method for unconstrained optimization.Dedicated to Phil Wolfe on his 65th birthday, in appreciation of his major contributions to mathematical programming.This material is based on research supported by Air Force Office of Scientific Research Grant AFOSR-89-0410 and National Science Foundation Grant CCR-9101801.     

Hyperbolic smoothing function method for minimax problems

In this article, an approach for solving finite minimax problems is proposed. This approach is based on the use of hyperbolic smoothing functions. In order to apply the hyperbolic smoothing we reformulate the objective function in the minimax problem and study the relationship between the original minimax and reformulated problems. We also study main properties of the hyperbolic smoothing function. Based on these results an algorithm for solving the finite minimax problem is proposed and this algorithm is implemented in general algebraic modelling system. We present preliminary results of numerical experiments with well-known nonsmooth optimization test problems. We also compare the proposed algorithm with the algorithm that uses the exponential smoothing function as well as with the algorithm based on nonlinear programming reformulation of the finite minimax problem.     

A numerical method for solving stochastic programming problems with moment constraints on a distribution function

The stochastic programming problem is considered in the case of a distribution function with partially known random parameters. A minimax approach is taken, and a numerical method is proposed for problems when information on the distribution function can be expressed in the form of finitely many moment constraints. Convergence is proved and results of numerical experiments are reported.     

Generalized fractional programming duality: A parametric approach

Using a parametric approach, duality is presented for a minimax fractional programming problem that involves several ratios in the objective function.The first author is thankful to Natural Science and Engineering Research Council of Canada for financial support through Grant A-5319, and the authors are thankful to the anonymous referees for useful suggestions.     

Corrected sequential linear programming for sparse minimax optimization

We present a new algorithm for nonlinear minimax optimization which is well suited for large and sparse problems. The method is based on trust regions and sequential linear programming. On each iteration a linear minimax problem is solved for a basic step. If necessary, this is followed by the determination of a minimum norm corrective step based on a first-order Taylor approximation. No Hessian information needs to be stored. Global convergence is proved. This new method has been extensively tested and compared with other methods, including two well known codes for nonlinear programming. The numerical tests indicate that in many cases the new method can find the solution in just as few iterations as methods based on approximate second-order information. The tests also show that for some problems the corrective steps give much faster convergence than for similar methods which do not employ such steps.Research supported partly by The Nordic Council of Ministers, The Icelandic Science Council, The University of Iceland Research Fund and The Danish Science Research Council.     

Towards Strong Duality in Integer Programming

We consider in this paper the Lagrangian dual method for solving general integer programming. New properties of Lagrangian duality are derived by a means of perturbation analysis. In particular, a necessary and sufficient condition for a primal optimal solution to be generated by the Lagrangian relaxation is obtained. The solution properties of Lagrangian relaxation problem are studied systematically. To overcome the difficulties caused by duality gap between the primal problem and the dual problem, we introduce an equivalent reformulation for the primal problem via applying a pth power to the constraints. We prove that this reformulation possesses an asymptotic strong duality property. Primal feasibility and primal optimality of the Lagrangian relaxation problems can be achieved in this reformulation when the parameter p is larger than a threshold value, thus ensuring the existence of an optimal primal-dual pair. We further show that duality gap for this partial pth power reformulation is a strictly decreasing function of p in the case of a single constraint. Dedicated to Professor Alex Rubinov on the occasion of his 65th birthday. Research supported by the Research Grants Council of Hong Kong under Grant CUHK 4214/01E, and the National Natural Science Foundation of China under Grants 79970107 and 10571116.     

A barrier function method for minimax problems

This paper presents an algorithm based on barrier functions for solving semi-infinite minimax problems which arise in an engineering design setting. The algorithm bears a resemblance to some of the current interior penalty function methods used to solve constrained minimization problems. Global convergence is proven, and numerical results are reported which show that the algorithm is exceptionally robust, and that its performance is comparable, while its structure is simpler than that of current first-order minimax algorithms.This research was supported by the National Science Foundation grant ECS-8517362, the Air Force Office Scientific Research grant 86-0116, the California State MICRO program, and the United Kingdom Science and Engineering Research Council.     

Quadratic programming problems and related linear complementarity problems

This paper investigates the general quadratic programming problem, i.e., the problem of finding the minimum of a quadratic function subject to linear constraints. In the case where, over the set of feasible points, the objective function is bounded from below, this problem can be solved by the minimization of a linear function, subject to the solution set of a linear complementarity problem, representing the Kuhn-Tucker conditions of the quadratic problem.To detect in the quadratic problem the unboundedness from below of the objective function, necessary and sufficient conditions are derived. It is shown that, when these conditions are applied, the general quadratic programming problem becomes equivalent to the investigation of an appropriately formulated linear complementarity problem.This research was supported by the Hungarian Research Foundation, Grant No. OTKA/1044.     

New approach for nonseparable dynamic programming problems

A general class of nonseparable dynamic problems is studied in a dynamic programming framework by introducingkth-order separability. The solution approach uses multiobjective dynamic programming as a separation strategy forkth-order separable dynamic problems. The theoretical grounding on which the optimal solution of the original nonseparable dynamic problem can be attained by a noninferior solution of the corresponding multiobjective dynamic programming problem is established. The relationship between the overall optimal Lagrangian multipliers and the stage-optimal Lagrangian multipliers and the relationship between the overall weighting vector and the stage weighting vector are explored, providing the basis for identifying the optimal solution of the original nonseparable problem from among the set of noninferior solutions generated by the envelope approach.This work was supported in part by NSF Grant No. CES-86-17984. The authors appreciate the comments from Dr. V. Chankong and the editorial work by Mrs. V. Benade and Dr. S. Hitchcock.