Properties of saddle points for generalized augmented Lagrangian

For inequality constrained optimization problem, we show the existence of local saddle point of generalized augmented Lagrangian under weak second-order sufficient conditions which are weaker than the second-order sufficient conditions in the literature. We further discuss the existence of global saddle points without requiring the uniqueness of the global optimal solution. This work was supported by the National Natural Science Foundation of China grants 10571106, 10471159.     

Exact augmented Lagrangian functions for nonlinear semidefinite programming

In this paper, we study augmented Lagrangian functions for nonlinear semidefinite programming (NSDP) problems with exactness properties. The term exact is used in the sense that the penalty parameter can be taken appropriately, so a single minimization of the augmented Lagrangian recovers a solution of the original problem. This leads to reformulations of NSDP problems into unconstrained nonlinear programming ones. Here, we first establish a unified framework for constructing these exact functions, generalizing Di Pillo and Lucidi’s work from 1996, that was aimed at solving nonlinear programming problems. Then, through our framework, we propose a practical augmented Lagrangian function for NSDP, proving that it is continuously differentiable and exact under the so-called nondegeneracy condition. We also present some preliminary numerical experiments.     

Duality in nonlinear programs using augmented Lagrangian functions

A generally nonconvex optimization problem with equality constraints is studied. The problem is introduced as an “inf sup” of a generalized augmented Lagrangian function. A dual problem is defined as the “sup inf” of the same generalized augmented Lagrangian. Sufficient conditions are derived for constructing the augmented Lagrangian function such that the extremal values of the primal and dual problems are equal. Characterization of a class of augmented Lagrangian functions which satisfy the sufficient conditions for strong duality is presented. Finally, some examples of functions and primal-dual problems in the above-mentioned class are presented.     

Duality in nonlinear multiobjective programming using augmented Lagrangian functions

A vector-valued generalized Lagrangian is constructed for a nonlinear multiobjective programming problem. Using the Lagrangian, a multiobjective dual is considered. Without assuming differentiability, weak and strong duality theorems are established using Pareto efficiency.The research of the second author was partially supported a GTE/SLU grant while visiting St. Lawrence University in the summer of 1991.     

Global convergence of modified augmented Lagrangian methods for nonlinear semidefinite programming

We investigate in this paper global convergence properties of the augmented Lagrangian method for nonlinear semidefinite programming (NLSDP). Four modified augmented Lagrangian methods for solving NLSDP based on different algorithmic strategies are proposed. Possibly infeasible limit points of the proposed methods are characterized. It is proved that feasible limit points that satisfy the Mangasarian-Fromovitz constraint qualification are KKT points of NLSDP without requiring the boundedness condition of the multipliers. Preliminary numerical results are reported to compare the performance of the modified augmented Lagrangian methods.     

Enclosure of all index-1 saddle points of general nonlinear functions

Transition states (index-1 saddle points) play a crucial role in determining the rates of chemical transformations but their reliable identification remains challenging in many applications. Deterministic global optimization methods have previously been employed for the location of transition states (TSs) by initially finding all stationary points and then identifying the TSs among the set of solutions. We propose several regional tests, applicable to general nonlinear, twice continuously differentiable functions, to accelerate the convergence of such approaches by identifying areas that do not contain any TS or that may contain a unique TS. The tests are based on the application of the interval extension of theorems from linear algebra to an interval Hessian matrix. They can be used within the framework of global optimization methods with the potential of reducing the computational time for TS location. We present the theory behind the tests, discuss their algorithmic complexity and show via a few examples that significant gains in computational time can be achieved by using these tests.     

A re-scaled twin augmented Lagrangian algorithm for saddle point seeking

In [A. Ouorou, A primal-dual algorithm for monotropic programming and its application to network optimization, Computational Optimization and Application 15 (2002) 125–143], a block-wise Gauss–Seidel method has been developed for monotropic programming problems, using two different quadratic augmented Lagrangian functions defined for the primal and the dual problems. In this paper, we extend the concept by introducing a nonlinear re-scaling principle obtained recently by Polyak [R. Polyak, Nonlinear rescaling vs smoothing technique in constrained optimization, Mathematical Programming 92 (2002) 197–235].     

Lagrangian duality and saddle points for sparse linear programming

The sparse linear programming(SLP) is a linear programming problem equipped with a sparsity constraint, which is nonconvex, discontinuous and generally NP-hard due to the combinatorial property involved.In this paper, by rewriting the sparsity constraint into a disjunctive form, we present an explicit formula of the Lagrangian dual problem for the SLP, in terms of an unconstrained piecewise-linear convex programming problem which admits a strong duality under bi-dual sparsity consistency. Furthermore, we show a saddle point theorem based on the strong duality and analyze two classes of stationary points for the saddle point problem. At last,we extend these results to SLP with the lower bound zero replaced by a certain negative constant.     

A nonlinear augmented Lagrangian for constrained minimax problems

A novel smooth nonlinear augmented Lagrangian for solving minimax problems with inequality constraints, is proposed in this paper, which has the positive properties that the classical Lagrangian and the penalty function fail to possess. The corresponding algorithm mainly consists of minimizing the nonlinear augmented Lagrangian function and updating the Lagrange multipliers and controlling parameter. It is demonstrated that the algorithm converges Q-superlinearly when the controlling parameter is less than a threshold under the mild conditions. Furthermore, the condition number of the Hessian of the nonlinear augmented Lagrangian function is studied, which is very important for the efficiency of the algorithm. The theoretical results are validated further by the preliminary numerical experiments for several testing problems reported at last, which show that the nonlinear augmented Lagrangian is promising.     

Convergence of augmented Lagrangian methods in extensions beyond nonlinear programming

Mathematical Programming - The augmented Lagrangian method (ALM) is extended to a broader-than-ever setting of generalized nonlinear programming in convex and nonconvex optimization that is capable...     

On the approximate augmented Lagrangian for nonlinear symmetric cone programming

This paper studies the approximate augmented Lagrangian for nonlinear symmetric cone programming. The analysis is based on some results under the framework of Euclidean Jordan algebras. We formulate the approximate Lagrangian dual problem and study conditions for approximate strong duality results and an approximate exact penalty representation. We also show, under Robinson’s constraint qualification, that the sequence of stationary points of the approximate augmented Lagrangian problems converges to a stationary point of the original nonlinear symmetric cone programming.     

On the convergence of augmented Lagrangian methods for nonlinear semidefinite programming

In this paper, we present new convergence properties of the augmented Lagrangian method for nonlinear semidefinite programs (NSDP). Convergence to the approximately global solutions and optimal values of NSDP is first established for a basic augmented Lagrangian scheme under mild conditions, without requiring the boundedness condition of the multipliers. We then propose four modified augmented Lagrangian methods for NSDP based on different algorithmic strategies. We show that the same convergence of the proposed methods can be ensured under weaker conditions.     

An augmented Lagrangian approach with a variable transformation in nonlinear programming

Tangent cone and (regular) normal cone of a closed set under an invertible variable transformation around a given point are investigated, which lead to the concepts of θ⁻¹-tangent cone of a set and θ⁻¹-subderivative of a function. When the notion of θ⁻¹-subderivative is applied to perturbation functions, a class of augmented Lagrangians involving an invertible mapping of perturbation variables are obtained, in which dualizing parameterization and augmenting functions are not necessarily convex in perturbation variables. A necessary and sufficient condition for the exact penalty representation under the proposed augmented Lagrangian scheme is obtained. For an augmenting function with an Euclidean norm, a sufficient condition (resp., a sufficient and necessary condition) for an arbitrary vector (resp., 0) to support an exact penalty representation is given in terms of θ⁻¹-subderivatives. An example of the variable transformation applied to constrained optimization problems is given, which yields several exact penalization results in the literature.     

Nonlinear separation approach for the augmented Lagrangian in nonlinear semidefinite programming

This paper aims at showing that the class of augmented Lagrangian functions for nonlinear semidefinite programming problems can be derived, as a particular case, from a nonlinear separation scheme in the image space associated with the given problem. By means of the image space analysis, a global saddle point condition for the augmented Lagrangian function is investigated. It is shown that the existence of a saddle point is equivalent to a regular nonlinear separation of two suitable subsets of the image space. Without requiring the strict complementarity, it is proved that, under second order sufficiency conditions, the augmented Lagrangian function admits a local saddle point. The existence of global saddle points is then obtained under additional assumptions that do not require the compactness of the feasible set. Motivated by the result on global saddle points, we propose two modified primal-dual methods based on the augmented Lagrangian using different strategies and prove their convergence to a global solution and the optimal value of the original problem without requiring the boundedness condition of the multiplier sequence.     

Minimax theorems and cone saddle points of uniformly same-order vector-valued functions

This paper is concerned with minimax theorems in vectorvalued optimization. A class of vector-valued functions which includes separated functionsf(x, y)=u(x)+v(y) as its proper subset is introduced. Minimax theorems and cone saddle-point theorems for this class of functions are investigated.The authors would like to thank two anonymous referees for helpful comments.     

A primal-dual augmented Lagrangian

Nonlinearly constrained optimization problems can be solved by minimizing a sequence of simpler unconstrained or linearly constrained subproblems. In this paper, we consider the formulation of subproblems in which the objective function is a generalization of the Hestenes-Powell augmented Lagrangian function. The main feature of the generalized function is that it is minimized with respect to both the primal and the dual variables simultaneously. The benefits of this approach include: (i) the ability to control the quality of the dual variables during the solution of the subproblem; (ii) the availability of improved dual estimates on early termination of the subproblem; and (iii) the ability to regularize the subproblem by imposing explicit bounds on the dual variables. We propose two primal-dual variants of conventional primal methods: a primal-dual bound constrained Lagrangian (pdBCL) method and a primal-dual ℓ ₁ linearly constrained Lagrangian (pdℓ ₁LCL) method. Finally, a new sequential quadratic programming (pdSQP) method is proposed that uses the primal-dual augmented Lagrangian as a merit function.     

A new augmented Lagrangian function for inequality constraints in nonlinear programming problems

In this paper, a new augmented Lagrangian function is introduced for solving nonlinear programming problems with inequality constraints. The relevant feature of the proposed approach is that, under suitable assumptions, it enables one to obtain the solution of the constrained problem by a single unconstrained minimization of a continuously differentiable function, so that standard unconstrained minimization techniques can be employed. Numerical examples are reported.     

A truncated Newton method in an augmented Lagrangian framework for nonlinear programming

In this paper we propose a primal-dual algorithm for the solution of general nonlinear programming problems. The core of the method is a local algorithm which relies on a truncated procedure for the computation of a search direction, and is thus suitable for large scale problems. The truncated direction produces a sequence of points which locally converges to a KKT pair with superlinear convergence rate.