Refinement of Lagrangian bounds in optimization problems

Lagrangian constraint relaxation and the corresponding bounds for the optimal value of an original optimization problem are examined. Techniques for the refinement of the classical Lagrangian bounds are investigated in the case where the complementary slackness conditions are not fulfilled because either the original formulation is nonconvex or the Lagrange multipliers are nonoptimal. Examples are given of integer and convex problems for which the modified bounds improve the classical Lagrangian bounds.     

Error analysis of variable degree mixed methods for elliptic problems via hybridization

A new approach to error analysis of hybridized mixed methods is proposed and applied to study a new hybridized variable degree Raviart-Thomas method for second order elliptic problems. The approach gives error estimates for the Lagrange multipliers without using error estimates for the other variables. Error estimates for the primal and flux variables then follow from those for the Lagrange multipliers. In contrast, traditional error analyses obtain error estimates for the flux and primal variables first and then use it to get error estimates for the Lagrange multipliers. The new approach not only gives new error estimates for the new variable degree Raviart-Thomas method, but also new error estimates for the classical uniform degree method with less stringent regularity requirements than previously known estimates. The error analysis is achieved by using a variational characterization of the Lagrange multipliers wherein the other unknowns do not appear. This approach can be applied to other hybridized mixed methods as well.

     

The infinite dimensional Lagrange multiplier rule for convex optimization problems

In this paper an infinite dimensional generalized Lagrange multipliers rule for convex optimization problems is presented and necessary and sufficient optimality conditions are given in order to guarantee the strong duality. Furthermore, an application is presented, in particular the existence of Lagrange multipliers associated to the bi-obstacle problem is obtained.     

Preparation of papers

Recent studies are concerned with two types of questions in nonconvex optimization: (a) conditions for having bounded Lagrange multipliers, Refs. 1–2; (b) a priori bounds for such Lagrange multipliers, Ref. 3. Such topics have been investigated under suitable regularity assumptions. The purpose of this paper is to study the same problems for the generalized Lagrange multipliers of a locally Lipschitz programming.The author thanks the referees for helpful suggestions     

On KKT points of Celis-Dennis-Tapia subproblem

The Celis-Dennis-Tapia(CDT) problem is a subproblem of the trust region algorithms for the constrained optimization. CDT subproblem is studied in this paper. It is shown that there exists the KKT point such that the Hessian matrix of the Lagrangian is positive semidefinite, if the multipliers at the global solution are not unique. Next the second order optimality conditions are also given, when the Hessian matrix of Lagrange at the solution has one negative eigenvalue. And furthermore, it is proved that all feasible KKT points satisfying that the corresponding Hessian matrices of Lagrange have one negative eigenvalue are the local optimal solutions of the CDT subproblem.     

Perturbation Lemma for the Newton Method with Application to the SQP Newton Method

We study the convergence of a general perturbation of the Newton method for solving a nonlinear system of equations. As an application, we show that the augmented Lagrangian successive quadratic programming is locally and q-quadratically convergent in the variable x to the solution of an equality constrained optimization problem, under a mild condition on the penalty parameter and the choice of the Lagrange multipliers.     

两个泛函优化定理的证明

在泛函优化理论中,Lagrange乘子定理、对偶定理占有重要地位.建立了带有等式和不等式约束的泛函优化问题,并给出了广义Lagrange乘子定理、广义Lagrange对偶定理的证明.     

New formulation of iterative substructuring methods without Lagrange multipliers: Neumann–Neumann and FETI

This article is devoted to introduce a new approach to iterative substructuring methods that, without recourse to Lagrange multipliers, yields positive definite preconditioned formulations of the Neumann–Neumann and FETI types. To my knowledge, this is the first time that such formulations have been made without resource to Lagrange multipliers. A numerical advantage that is concomitant to such multipliers‐free formulations is the reduction of the degrees of freedom associated with the Lagrange multipliers. Other attractive features are their generality, directness, and simplicity. The general framework of the new approach is rather simple and stems directly from the discretization procedures that are applied; in it, the differential operators act on discontinuous piecewise‐defined functions. Then, the Lagrange multipliers are not required because in such an environment the functions‐discontinuities are not an anomaly that need to be corrected. The resulting algorithms and equations‐systems are also derived with considerable detail. © 2007 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq, 2008     

Lagrange multipliers for set-valued optimization problems associated with coderivatives

In this paper we investigate a vector optimization problem (P) where objective and constraints are given by set-valued maps. We show that by mean of marginal functions and suitable scalarizing functions one can characterize certain solutions of (P) as solutions of a scalar optimization problem (SP) with single-valued objective and constraint functions. Then applying some classical or recent results in optimization theory to (SP) and using estimates of subdifferentials of marginal functions, we obtain optimality conditions for (P) expressed in terms of Lagrange or sequential Lagrange multipliers associated with various coderivatives of the set-valued data.     

Solving dual problems using a coevolutionary optimization algorithm

In solving certain optimization problems, the corresponding Lagrangian dual problem is often solved simply because in these problems the dual problem is easier to solve than the original primal problem. Another reason for their solution is the implication of the weak duality theorem which suggests that under certain conditions the optimal dual function value is smaller than or equal to the optimal primal objective value. The dual problem is a special case of a bilevel programming problem involving Lagrange multipliers as upper-level variables and decision variables as lower-level variables. Another interesting aspect of dual problems is that both lower and upper-level optimization problems involve only box constraints and no other equality of inequality constraints. In this paper, we propose a coevolutionary dual optimization (CEDO) algorithm for co-evolving two populations—one involving Lagrange multipliers and other involving decision variables—to find the dual solution. On 11 test problems taken from the optimization literature, we demonstrate the efficacy of CEDO algorithm by comparing it with a couple of nested smooth and nonsmooth algorithms and a couple of previously suggested coevolutionary algorithms. The performance of CEDO algorithm is also compared with two classical methods involving nonsmooth (bundle) optimization methods. As a by-product, we analyze the test problems to find their associated duality gap and classify them into three categories having zero, finite or infinite duality gaps. The development of a coevolutionary approach, revealing the presence or absence of duality gap in a number of commonly-used test problems, and efficacy of the proposed coevolutionary algorithm compared to usual nested smooth and nonsmooth algorithms and other existing coevolutionary approaches remain as the hallmark of the current study.     

A nonlinear Lagrangian for constrained optimization problems

A novel nonlinear Lagrangian is presented for constrained optimization problems with both inequality and equality constraints, which is nonlinear with respect to both functions in problem and Lagrange multipliers. The nonlinear Lagrangian inherits the smoothness of the objective and constraint functions and has positive properties. The algorithm on the nonlinear Lagrangian is demonstrated to possess local and linear convergence when the penalty parameter is less than a threshold (the penalty parameter in the penalty method has to approximate zero) under a set of suitable conditions, and be super-linearly convergent when the penalty parameter is decreased following Lagrange multiplier update. Furthermore, the dual problem based on the nonlinear Lagrangian is discussed and some important properties are proposed, which fail to hold for the dual problem based on the classical Lagrangian. At last, the preliminary and comparing numerical results for several typical test problems by using the new nonlinear Lagrangian algorithm and the other two related nonlinear Lagrangian algorithms, are reported, which show that the given nonlinear Lagrangian is promising.     

A Discrete Lagrangian-Based Global-Search Method for Solving Satisfiability Problems

Satisfiability is a class of NP-complete problems that model a wide range of real-world applications. These problems are difficult to solve because they have many local minima in their search space, often trapping greedy search methods that utilize some form of descent. In this paper, we propose a new discrete Lagrange-multiplier-based global-search method (DLM) for solving satisfiability problems. We derive new approaches for applying Lagrangian methods in discrete space, we show that an equilibrium is reached when a feasible assignment to the original problem is found and present heuristic algorithms to look for equilibrium points. Our method and analysis provides a theoretical foundation and generalization of local search schemes that optimize the objective alone and penalty-based schemes that optimize the constraints alone. In contrast to local search methods that restart from a new starting point when a search reaches a local trap, the Lagrange multipliers in DLM provide a force to lead the search out of a local minimum and move it in the direction provided by the Lagrange multipliers. In contrast to penalty-based schemes that rely only on the weights of violated constraints to escape from local minima, DLM also uses the value of an objective function (in this case the number of violated constraints) to provide further guidance. The dynamic shift in emphasis between the objective and the constraints, depending on their relative values, is the key of Lagrangian methods. One of the major advantages of DLM is that it has very few algorithmic parameters to be tuned by users. Besides the search procedure can be made deterministic and the results reproducible. We demonstrate our method by applying it to solve an extensive set of benchmark problems archived in DIMACS of Rutgers University. DLM often performs better than the best existing methods and can achieve an order-of-magnitude speed-up for some problems.     

Existence of augmented Lagrange multipliers for cone constrained optimization problems

In this paper, by using an augmented Lagrangian approach, we obtain several sufficient conditions for the existence of augmented Lagrange multipliers of a cone constrained optimization problem in Banach spaces, where the corresponding augmenting function is assumed to have a valley at zero. Furthermore, we deal with the relationship of saddle points, augmented Lagrange multipliers, and zero duality gap property between the cone constrained optimization problem and its augmented Lagrangian dual problem.     

On constraint qualifications for second-order optimality conditions depending on a single Lagrange multiplier

We present new constraint qualifications (CQs) to ensure the validity of some well-known second-order optimality conditions. Our main interest is on second-order conditions that can be associated with numerical methods for solving constrained optimization problems. Such conditions depend on a single Lagrange multiplier, instead of the whole set of Lagrange multipliers. For each condition, we characterize the weakest CQ that guarantees its fulfillment at local minimizers, while proposing new weak conditions implying them. Relations with other CQs are discussed.     

A Global Optimization Algorithm using Lagrangian Underestimates and the Interval Newton Method

Convex relaxations can be used to obtain lower bounds on the optimal objective function value of nonconvex quadratically constrained quadratic programs. However, for some problems, significantly better bounds can be obtained by minimizing the restricted Lagrangian function for a given estimate of the Lagrange multipliers. The difficulty in utilizing Lagrangian duality within a global optimization context is that the restricted Lagrangian is often nonconvex. Minimizing a convex underestimate of the restricted Lagrangian overcomes this difficulty and facilitates the use of Lagrangian duality within a global optimization framework. A branch-and-bound algorithm is presented that relies on these Lagrangian underestimates to provide lower bounds and on the interval Newton method to facilitate convergence in the neighborhood of the global solution. Computational results show that the algorithm compares favorably to the Reformulation–Linearization Technique for problems with a favorable structure.     

Lagrange optimality system for a class of nonsmooth convex optimization

In this paper, we revisit the augmented Lagrangian method for a class of nonsmooth convex optimization. We present the Lagrange optimality system of the augmented Lagrangian associated with the problems, and establish its connections with the standard optimality condition and the saddle point condition of the augmented Lagrangian, which provides a powerful tool for developing numerical algorithms: we derive a Lagrange–Newton algorithm for the nonsmooth convex optimization, and establish the nonsingularity of the Newton system and the local convergence of the algorithm.     

对等式约束非线性规划问题的Hestenes-Powell增广拉格朗日函数的进一步研究

本文对用无约束极小化方法求解等式约束非线性规划问题的Hestenes-Powell 增广拉格朗日函数作了进一步研究．在适当的条件下,我们建立了Hestenes-Powell增广拉格朗日函数在原问题变量空间上的无约束极小与原约束问题的解之间的关系,并且也给出了Hestenes-Powell增广拉格朗日函数在原问题变量和乘子变量的积空间上的无约束极小与原约束问题的解之间的一个关系．因此,从理论的观点来看,原约束问题的解和对应的拉格朗日乘子值不仅可以用众所周知的乘子法求得,而且可以通过对Hestenes-Powell 增广拉格朗日函数在原问题变量和乘子变量的积空间上执行一个单一的无约束极小化来获得．     

Two-level primal-dual decomposition technique for large-scale nonconvex optimization problems with constraints

A two-level decomposition method for nonconvex separable optimization problems with additional local constraints of general inequality type is presented and thoroughly analyzed in the paper. The method is of primal-dual type, based on an augmentation of the Lagrange function. Previous methods of this type were in fact three-level, with adjustment of the Lagrange multipliers at one of the levels. This level is eliminated in the present approach by replacing the multipliers by a formula depending only on primal variables and Kuhn-Tucker multipliers for the local constraints. The primal variables and the Kuhn-Tucker multipliers are together the higher-level variables, which are updated simultaneously. Algorithms for this updating are proposed in the paper, together with their convergence analysis, which gives also indications on how to choose penalty coefficients of the augmented Lagrangian. Finally, numerical examples are presented.     

Optimization with a class of multivariate integral stochastic order constraints

We study convex optimization problems with a class of multivariate integral stochastic order constraints defined in terms of parametrized families of increasing concave functions. We show that utility functions act as the Lagrange multipliers of the stochastic order constraints in this general setting, and that the dual problem is a search over utility functions. Practical implementation issues are discussed.     

Lagrange Multiplier Conditions Characterizing the Optimal Solution Sets of Cone-Constrained Convex Programs

Various characterizations of optimal solution sets of cone-constrained convex optimization problems are given. The results are expressed in terms of subgradients and Lagrange multipliers. We establish first that the Lagrangian function of a convex program is constant on the optimal solution set. This elementary property is then used to derive various simple Lagrange multiplier-based characterizations of the solution set. For a finite-dimensional convex program with inequality constraints, the characterizations illustrate that the active constraints with positive Lagrange multipliers at an optimal solution remain active at all optimal solutions of the program. The results are applied to derive corresponding Lagrange multiplier characterizations of the solution sets of semidefinite programs and fractional programs. Specific examples are given to illustrate the nature of the results.