Duality and Penalization in Optimization via an Augmented Lagrangian Function with Applications

This paper aims to establish duality and exact penalization results for the primal problem of minimizing an extended real-valued function in a reflexive Banach space in terms of a valley-at-0 augmented Lagrangian function. It is shown that every weak limit point of a sequence of optimal solutions generated by the valley-at-0 augmented Lagrangian problems is a solution of the original problem. A zero duality gap property and an exact penalization representation between the primal problem and the valley-at-0 augmented Lagrangian dual problem are obtained. These results are then applied to an inequality and equality constrained optimization problem in infinite-dimensional spaces and variational problems in Sobolev spaces, respectively. The first author was supported by the Research Committee of Hong Kong Polytechnic University, by Grant 10571174 from the National Natural Science Foundation of China and Grant 08KJB11009 from the Jiangsu Education Committee of China. The second author was supported by Grant BQ771 from the Research Grants Council of Hong Kong. We are grateful to the referees for useful suggestions which have contributed to the final presentation of the paper.     

Augmented Lagrangian Objective Penalty Function

Augmented Lagrangian function is one of the most important tools used in solving some constrained optimization problems. In this article, we study an augmented Lagrangian objective penalty function and a modified augmented Lagrangian objective penalty function for inequality constrained optimization problems. First, we prove the dual properties of the augmented Lagrangian objective penalty function, which are at least as good as the traditional Lagrangian function's. Under some conditions, the saddle point of the augmented Lagrangian objective penalty function satisfies the first-order Karush-Kuhn-Tucker condition. This is especially so when the Karush-Kuhn-Tucker condition holds for convex programming of its saddle point existence. Second, we prove the dual properties of the modified augmented Lagrangian objective penalty function. For a global optimal solution, when the exactness of the modified augmented Lagrangian objective penalty function holds, its saddle point exists. The sufficient and necessary stability conditions used to determine whether the modified augmented Lagrangian objective penalty function is exact for a global solution is proved. Based on the modified augmented Lagrangian objective penalty function, an algorithm is developed to find a global solution to an inequality constrained optimization problem, and its global convergence is also proved under some conditions. Furthermore, the sufficient and necessary calmness condition on the exactness of the modified augmented Lagrangian objective penalty function is proved for a local solution. An algorithm is presented in finding a local solution, with its convergence proved under some conditions.     

Augmented Lagrangian functions for constrained optimization problems

In this paper, in order to obtain some existence results about solutions of the augmented Lagrangian problem for a constrained problem in which the objective function and constraint functions are noncoercive, we construct a new augmented Lagrangian function by using an auxiliary function. We establish a zero duality gap result and a sufficient condition of an exact penalization representation for the constrained problem without the coercive or level-bounded assumption on the objective function and constraint functions. By assuming that the sequence of multipliers is bounded, we obtain the existence of a global minimum and an asymptotically minimizing sequence for the constrained optimization problem.     

Some Results about Duality and Exact Penalization

In this paper, we introduce the concept of the valley at 0 augmenting function and apply it to construct a class of valley at 0 augmented Lagrangian functions. We establish the existence of a path of optimal solutions generated by valley at 0 augmented Lagrangian problems and its convergence toward the optimal set of the original problem and obtain the zero duality gap property between the primal problem and the valley at 0 augmented Lagrangian dual problem. Moreover, we establish the exact penalization representation results in the framework of valley at 0 augmented Lagrangian.     

Further Study on Augmented Lagrangian Duality Theory

In this paper, we present a necessary and sufficient condition for a zero duality gap between a primal optimization problem and its generalized augmented Lagrangian dual problems. The condition is mainly expressed in the form of the lower semicontinuity of a perturbation function at the origin. For a constrained optimization problem, a general equivalence is established for zero duality gap properties defined by a general nonlinear Lagrangian dual problem and a generalized augmented Lagrangian dual problem, respectively. For a constrained optimization problem with both equality and inequality constraints, we prove that first-order and second-order necessary optimality conditions of the augmented Lagrangian problems with a convex quadratic augmenting function converge to that of the original constrained program. For a mathematical program with only equality constraints, we show that the second-order necessary conditions of general augmented Lagrangian problems with a convex augmenting function converge to that of the original constrained program.This research is supported by the Research Grants Council of Hong Kong (PolyU B-Q359.)     

Augmented Lagrangian function, non-quadratic growth condition and exact penalization

In this paper, under the assumption that the perturbation function satisfies a growth condition, necessary and sufficient conditions for an exact penalty representation and a zero duality gap property between the primal problem and its augmented Lagrangian dual problem are established.     

Duality Theory in Fuzzy Optimization Problems

A solution concept of fuzzy optimization problems, which is essentially similar to the notion of Pareto optimal solution (nondominated solution) in multiobjective programming problems, is introduced by imposing a partial ordering on the set of all fuzzy numbers. We also introduce a concept of fuzzy scalar (inner) product based on the positive and negative parts of fuzzy numbers. Then the fuzzy-valued Lagrangian function and the fuzzy-valued Lagrangian dual function for the fuzzy optimization problem are proposed via the concept of fuzzy scalar product. Under these settings, the weak and strong duality theorems for fuzzy optimization problems can be elicited. We show that there is no duality gap between the primal and dual fuzzy optimization problems under suitable assumptions for fuzzy-valued functions.     

Bundle Relaxation and Primal Recovery in Unit Commitment Problems. The Brazilian Case

We consider the inclusion of commitment of thermal generation units in the optimal management of the Brazilian power system. By means of Lagrangian relaxation we decompose the problem and obtain a nondifferentiable dual function that is separable. We solve the dual problem with a bundle method. Our purpose is twofold: first, bundle methods are the methods of choice in nonsmooth optimization when it comes to solve large-scale problems with high precision. Second, they give good starting points for recovering primal solutions. We use an inexact augmented Lagrangian technique to find a near-optimal primal feasible solution. We assess our approach with numerical results.     

An Exact Augmented Lagrangian Function for Nonlinear Programming with Two-Sided Constraints

This paper is aimed toward the definition of a new exact augmented Lagrangian function for two-sided inequality constrained problems. The distinguishing feature of this augmented Lagrangian function is that it employs only one multiplier for each two-sided constraint. We prove that stationary points, local minimizers and global minimizers of the exact augmented Lagrangian function correspond exactly to KKT pairs, local solutions and global solutions of the constrained problem.     

Duality theory in multiobjective programming

In this paper, a multiobjective programming problem is considered as that of finding the set of all nondominated solutions with respect to the given domination cone. Two point-to-set maps, the primal map and the dual map, and the vector-valued Lagrangian function are defined, corresponding to the case of a scalar optimization problem. The Lagrange multiplier theorem, the saddle-point theorem, and the duality theorem are derived by using the properties of these maps under adequate convexity assumptions and regularity conditions.     

Duality and Exact Penalization for General Augmented Lagrangians

We consider a problem of minimizing an extended real-valued function defined in a Hausdorff topological space. We study the dual problem induced by a general augmented Lagrangian function. Under a simple set of assumptions on this general augmented Lagrangian function, we obtain strong duality and existence of exact penalty parameter via an abstract convexity approach. We show that every cluster point of a sub-optimal path related to the dual problem is a primal solution. Our assumptions are more general than those recently considered in the related literature.     

Duality in nonlinear programs using augmented Lagrangian functions

A generally nonconvex optimization problem with equality constraints is studied. The problem is introduced as an “inf sup” of a generalized augmented Lagrangian function. A dual problem is defined as the “sup inf” of the same generalized augmented Lagrangian. Sufficient conditions are derived for constructing the augmented Lagrangian function such that the extremal values of the primal and dual problems are equal. Characterization of a class of augmented Lagrangian functions which satisfy the sufficient conditions for strong duality is presented. Finally, some examples of functions and primal-dual problems in the above-mentioned class are presented.     

向量优化中广义增广拉格朗日对偶理论及应用

作者介绍了一种基于向量值延拓函数的广义增广拉格朗日函数,建立了基于广义增广拉格朗日函数的集值广义增广拉格朗日对偶映射和相应的对偶问题,得到了相应的强对偶和弱对偶结果,将所获结果应用到约束向量优化问题.该文的结果推广了一些已有的结论.     

A geometric framework for nonconvex optimization duality using augmented lagrangian functions

We provide a unifying geometric framework for the analysis of general classes of duality schemes and penalty methods for nonconvex constrained optimization problems. We present a separation result for nonconvex sets via general concave surfaces. We use this separation result to provide necessary and sufficient conditions for establishing strong duality between geometric primal and dual problems. Using the primal function of a constrained optimization problem, we apply our results both in the analysis of duality schemes constructed using augmented Lagrangian functions, and in establishing necessary and sufficient conditions for the convergence of penalty methods.     

The exact penalty principle

The exact penalty approach aims at replacing a constrained optimization problem by an equivalent unconstrained optimization problem. Most results in the literature of exact penalization are mainly concerned with finding conditions under which a solution of the constrained optimization problem is a solution of an unconstrained penalized optimization problem, and the reverse property is rarely studied. In this paper, we study the reverse property. We give the conditions under which the original constrained (single and/or multiobjective) optimization problem and the unconstrained exact penalized problem are exactly equivalent. The main conditions to ensure the exact penalty principle for optimization problems include the global and local error bound conditions. By using variational analysis, these conditions may be characterized by using generalized differentiation.     

Existence of augmented Lagrange multipliers for cone constrained optimization problems

In this paper, by using an augmented Lagrangian approach, we obtain several sufficient conditions for the existence of augmented Lagrange multipliers of a cone constrained optimization problem in Banach spaces, where the corresponding augmenting function is assumed to have a valley at zero. Furthermore, we deal with the relationship of saddle points, augmented Lagrange multipliers, and zero duality gap property between the cone constrained optimization problem and its augmented Lagrangian dual problem.     

Nonlinear Lagrangian Theory for Nonconvex Optimization

The Lagrangian function in the conventional theory for solving constrained optimization problems is a linear combination of the cost and constraint functions. Typically, the optimality conditions based on linear Lagrangian theory are either necessary or sufficient, but not both unless the underlying cost and constraint functions are also convex.We propose a somewhat different approach for solving a nonconvex inequality constrained optimization problem based on a nonlinear Lagrangian function. This leads to optimality conditions which are both sufficient and necessary, without any convexity assumption. Subsequently, under appropriate assumptions, the optimality conditions derived from the new nonlinear Lagrangian approach are used to obtain an equivalent root-finding problem. By appropriately defining a dual optimization problem and an alternative dual problem, we show that zero duality gap will hold always regardless of convexity, contrary to the case of linear Lagrangian duality.     

Augmented Lagrangian Methods for $p$-Harmonic Flows with the Generalized Penalization Terms and Application to Image Processing

In this paper, we propose a generalized penalization technique and a convex constraint minimization approach for the $p$-harmonic flow problem following the ideas in [Kang & March, IEEE T. Image Process., 16 (2007), 2251-2261]. We use fast algorithms to solve the subproblems, such as the dual projection methods, primal-dual methods and augmented Lagrangian methods. With a special penalization term, some special algorithms are presented. Numerical experiments are given to demonstrate the performance of the proposed methods. We successfully show that our algorithms are effective and efficient due to two reasons: the solver for subproblem is fast in essence and there is no need to solve the subproblem accurately (even 2 inner iterations of the subproblem are enough). It is also observed that better PSNR values are produced using the new algorithms.     

Log-Sigmoid Multipliers Method in Constrained Optimization

In this paper we introduced and analyzed the Log-Sigmoid (LS) multipliers method for constrained optimization. The LS method is to the recently developed smoothing technique as augmented Lagrangian to the penalty method or modified barrier to classical barrier methods. At the same time the LS method has some specific properties, which make it substantially different from other nonquadratic augmented Lagrangian techniques.We established convergence of the LS type penalty method under very mild assumptions on the input data and estimated the rate of convergence of the LS multipliers method under the standard second order optimality condition for both exact and nonexact minimization.Some important properties of the dual function and the dual problem, which are based on the LS Lagrangian, were discovered and the primal–dual LS method was introduced.     

Approximate Augmented Lagrangian Functions and Nonlinear Semidefinite Programs

In this paper, an approximate augmented Lagrangian function for nonlinear semidefinite programs is introduced. Some basic properties of the approximate augmented Lagrange function such as monotonicity and convexity are discussed. Necessary and sufficient conditions for approximate strong duality results are derived. Conditions for an approximate exact penalty representation in the framework of augmented Lagrangian are given. Under certain conditions, it is shown that any limit point of a sequence of stationary points of approximate augmented Lagrangian problems is a KKT point of the original semidefinite program and that a sequence of optimal solutions to augmented Lagrangian problems converges to a solution of the original semidefinite program.