Stabilized sequential quadratic programming for optimization and a stabilized Newton-type method for variational problems

The stabilized version of the sequential quadratic programming algorithm (sSQP) had been developed in order to achieve fast convergence despite possible degeneracy of constraints of optimization problems, when the Lagrange multipliers associated to a solution are not unique. Superlinear convergence of sSQP had been previously established under the strong second-order sufficient condition for optimality (without any constraint qualification assumptions). We prove a stronger superlinear convergence result than the above, assuming the usual second-order sufficient condition only. In addition, our analysis is carried out in the more general setting of variational problems, for which we introduce a natural extension of sSQP techniques. In the process, we also obtain a new error bound for Karush–Kuhn–Tucker systems for variational problems that holds under an appropriate second-order condition.     

Some theoretical limitations of second-order algorithms for smooth constrained optimization

In second-order algorithms, we investigate the relevance of the constant rank of the full set of active constraints in ensuring global convergence to a second-order stationary point under a constraint qualification. We show that second-order stationarity is not expected in the non-constant rank case if the growth of so-called tangent AKKT2 sequences is not controlled. Since no algorithm controls their growth, we argue that there is a theoretical limitation of algorithms in finding second-order stationary points without constant rank assumptions.     

On constraint qualifications for second-order optimality conditions depending on a single Lagrange multiplier

We present new constraint qualifications (CQs) to ensure the validity of some well-known second-order optimality conditions. Our main interest is on second-order conditions that can be associated with numerical methods for solving constrained optimization problems. Such conditions depend on a single Lagrange multiplier, instead of the whole set of Lagrange multipliers. For each condition, we characterize the weakest CQ that guarantees its fulfillment at local minimizers, while proposing new weak conditions implying them. Relations with other CQs are discussed.     

Stabilized SQP revisited

The stabilized version of the sequential quadratic programming algorithm (sSQP) had been developed in order to achieve superlinear convergence in situations when the Lagrange multipliers associated to a solution are not unique. Within the framework of Fischer (Math Program 94:91–124, 2002), the key to local superlinear convergence of sSQP are the following two properties: upper Lipschitzian behavior of solutions of the Karush-Kuhn-Tucker (KKT) system under canonical perturbations and local solvability of sSQP subproblems with the associated primal-dual step being of the order of the distance from the current iterate to the solution set of the unperturbed KKT system. According to Fernández and Solodov (Math Program 125:47–73, 2010), both of these properties are ensured by the second-order sufficient optimality condition (SOSC) without any constraint qualification assumptions. In this paper, we state precise relationships between the upper Lipschitzian property of solutions of KKT systems, error bounds for KKT systems, the notion of critical Lagrange multipliers (a subclass of multipliers that violate SOSC in a very special way), the second-order necessary condition for optimality, and solvability of sSQP subproblems. Moreover, for the problem with equality constraints only, we prove superlinear convergence of sSQP under the assumption that the dual starting point is close to a noncritical multiplier. Since noncritical multipliers include all those satisfying SOSC but are not limited to them, we believe this gives the first superlinear convergence result for any Newtonian method for constrained optimization under assumptions that do not include any constraint qualifications and are weaker than SOSC. In the general case when inequality constraints are present, we show that such a relaxation of assumptions is not possible. We also consider applying sSQP to the problem where inequality constraints are reformulated into equalities using slack variables, and discuss the assumptions needed for convergence in this approach. We conclude with consequences for local regularization methods proposed in (Izmailov and Solodov SIAM J Optim 16:210–228, 2004; Wright SIAM J. Optim. 15:673–676, 2005). In particular, we show that these methods are still locally superlinearly convergent under the noncritical multiplier assumption, weaker than SOSC employed originally.     

On attraction of Newton-type iterates to multipliers violating second-order sufficiency conditions

Assuming that the primal part of the sequence generated by a Newton-type (e.g., SQP) method applied to an equality-constrained problem converges to a solution where the constraints are degenerate, we investigate whether the dual part of the sequence is attracted by those Lagrange multipliers which satisfy second-order sufficient condition (SOSC) for optimality, or by those multipliers which violate it. This question is relevant at least for two reasons: one is speed of convergence of standard methods; the other is applicability of some recently proposed approaches for handling degenerate constraints. We show that for the class of damped Newton methods, convergence of the dual sequence to multipliers satisfying SOSC is unlikely to occur. We support our findings by numerical experiments. We also suggest a simple auxiliary procedure for computing multiplier estimates, which does not have this undesirable property. Finally, some consequences for the case of mixed equality and inequality constraints are discussed.     

Abadie-Type Constraint Qualification for Mathematical Programs with Equilibrium Constraints

Mathematical programs with equilibrium constraints (MPEC) are nonlinear programs which do not satisfy any of the common constraint qualifications (CQ). In order to obtain first-order optimality conditions, constraint qualifications tailored to the MPECs have been developed and researched in the past. In this paper, we introduce a new Abadie-type constraint qualification for MPECs. We investigate sufficient conditions for this new CQ, discuss its relationship to several existing MPEC constraint qualifications, and introduce a new Slater-type constraint qualifications. Finally, we prove a new stationarity concept to be a necessary optimality condition under our new Abadie-type CQ.Communicated by Z. Q. Luo     

变序结构局部弱非控点的二阶刻画

引进了一种二阶切导数,借助该切导数给出了变序结构集值优化问题取得局部弱非控点的二阶最优性必要条件.在某种特殊情况下,给出了一阶最优性条件.通过修正的Dubovitskij-Miljutin切锥导出的约束规格,给出了两个集值映射之和的二阶相依切导数的关系式,进一步得到目标函数与变锥函数的二阶相依切导数分开形式的最优性必要条件.     

Inexact Josephy–Newton framework for generalized equations and its applications to local analysis of Newtonian methods for constrained optimization

We propose and analyze a perturbed version of the classical Josephy–Newton method for solving generalized equations. This perturbed framework is convenient to treat in a unified way standard sequential quadratic programming, its stabilized version, sequential quadratically constrained quadratic programming, and linearly constrained Lagrangian methods. For the linearly constrained Lagrangian methods, in particular, we obtain superlinear convergence under the second-order sufficient optimality condition and the strict Mangasarian–Fromovitz constraint qualification, while previous results in the literature assume (in addition to second-order sufficiency) the stronger linear independence constraint qualification as well as the strict complementarity condition. For the sequential quadratically constrained quadratic programming methods, we prove primal-dual superlinear/quadratic convergence under the same assumptions as above, which also gives a new result.     

On Weak and Strong Kuhn–Tucker Conditions for Smooth Multiobjective Optimization

We consider a smooth multiobjective optimization problem with inequality constraints. Weak Kuhn?CTucker (WKT) optimality conditions are said to hold for such problems when not all the multipliers of the objective functions are zero, while strong Kuhn?CTucker (SKT) conditions are said to hold when all the multipliers of the objective functions are positive. We introduce a new regularity condition under which (WKT) hold. Moreover, we prove that for another new regularity condition (SKT) hold at every Geoffrion-properly efficient point. We show with an example that the assumption on proper efficiency cannot be relaxed. Finally, we prove that Geoffrion-proper efficiency is not needed when the constraint set is polyhedral and the objective functions are linear.     

The regularized feasible directions method for nonconvex optimization

This paper develops and studies a feasible directions approach for the minimization of a continuous function over linear constraints in which the update directions belong to a predetermined finite set spanning the feasible set. These directions are recurrently investigated in a cyclic semi-random order, where the stepsize of the update is determined via univariate optimization. We establish that any accumulation point of this optimization procedure is a stationary point of the problem, meaning that the directional derivative in any feasible direction is nonnegative. To assess and establish a rate of convergence, we develop a new optimality measure that acts as a proxy for the stationarity condition, and substantiate its role by showing that it is coherent with first-order conditions in specific scenarios. Finally we prove that our method enjoys a sublinear rate of convergence of this optimality measure in expectation.     

Second-order conditions for nonsmooth multiobjective optimization problems with inclusion constraints

This paper investigates second-order optimality conditions for general multiobjective optimization problems with constraint set-valued mappings and an arbitrary constraint set in Banach spaces. Without differentiability nor convexity on the data and with a metric regularity assumption the second-order necessary conditions for weakly efficient solutions are given in the primal form. Under some additional assumptions and with the help of Robinson -Ursescu open mapping theorem we obtain dual second-order necessary optimality conditions in terms of Lagrange-Kuhn-Tucker multipliers. Also, the second-order sufficient conditions are established whenever the decision space is finite dimensional. To this aim, we use the second-order projective derivatives associated to the second-order projective tangent sets to the graphs introduced by Penot. From the results obtained in this paper, we deduce and extend, in the special case some known results in scalar optimization and improve substantially the few results known in vector case.     

集值优化强有效解的广义二阶锥方向导数刻画

在实赋范线性空间中考虑集值优化问题的强有效性．借助Henig扩张锥和基泛函的性质,利用广义二阶锥方向相依导数,得到受约束于集值映射的优化问题,取得强有效元的二阶最优性必要条件．当目标函数为近似锥一次类凸映射时,利用强有效点的标量化定理,得到集值优化问题,取得强有效元的二阶充分条件．     

Metric regularity,tangent sets,and second-order optimality conditions

A strong regularity theorem is proved, which shows that the usual constraint qualification conditions ensuring the regularity of the set-valued maps expressing feasibility in optimization problems, are in fact minimal assumptions. These results are then used to derive calculus rules for second-order tangent sets, allowing us in turn to obtain a second-order (Lagrangian) necessary condition for optimality which completes the usual one of positive semidefiniteness on the Hessian of the Lagrangian function.On leave from Universidad de Chile, Casilla 170/3 Correo 3, Santiago, Chile.     

A New Trust-Region Algorithm for Equality Constrained Optimization

We present a new trust-region algorithm for solving nonlinear equality constrained optimization problems. Quadratic penalty functions are employed to obtain global convergence. At each iteration a local change of variables is performed to improve the ability of the algorithm to follow the constraint level set. Under certain assumptions we prove that this algorithm globally converges to a point satisfying the second-order necessary optimality conditions. Results of preliminary numerical experiments are reported.     

一类新的二阶组合切导数及其应用

引进了一种新的切锥,讨论它与相依切锥的关系.借助这种新的切锥引进了一类新的二阶组合切导数,并讨论了它与其他二阶切导数的关系.利用这类新的二阶组合切导数,建立了集值优化分别取得Henig有效元和全局有效元的最优性必要条件.     

Enhanced Karush–Kuhn–Tucker condition and weaker constraint qualifications

In this paper we study necessary optimality conditions for nonsmooth optimization problems with equality, inequality and abstract set constraints. We derive the enhanced Fritz John condition which contains some new information even in the smooth case than the classical enhanced Fritz John condition. From this enhanced Fritz John condition we derive the enhanced Karush–Kuhn–Tucker condition and introduce the associated pseudonormality and quasinormality condition. We prove that either pseudonormality or quasinormality with regularity on the constraint functions and the set constraint implies the existence of a local error bound. Finally we give a tighter upper estimate for the Fréchet subdifferential and the limiting subdifferential of the value function in terms of quasinormal multipliers which is usually a smaller set than the set of classical normal multipliers. In particular we show that the value function of a perturbed problem is Lipschitz continuous under the perturbed quasinormality condition which is much weaker than the classical normality condition.     

On local convergence of sequential quadratically-constrained quadratic-programming type methods,with an extension to variational problems

We consider the class of quadratically-constrained quadratic-programming methods in the framework extended from optimization to more general variational problems. Previously, in the optimization case, Anitescu (SIAM J. Optim. 12, 949–978, 2002) showed superlinear convergence of the primal sequence under the Mangasarian-Fromovitz constraint qualification and the quadratic growth condition. Quadratic convergence of the primal-dual sequence was established by Fukushima, Luo and Tseng (SIAM J. Optim. 13, 1098–1119, 2003) under the assumption of convexity, the Slater constraint qualification, and a strong second-order sufficient condition. We obtain a new local convergence result, which complements the above (it is neither stronger nor weaker): we prove primal-dual quadratic convergence under the linear independence constraint qualification, strict complementarity, and a second-order sufficiency condition. Additionally, our results apply to variational problems beyond the optimization case. Finally, we provide a necessary and sufficient condition for superlinear convergence of the primal sequence under a Dennis-Moré type condition. Research of the second author is partially supported by CNPq Grants 300734/95-6 and 471780/2003-0, by PRONEX–Optimization, and by FAPERJ.     

Local stability of solutions to differentiable optimization problems in banach spaces

This paper considers a class of nonlinear differentiable optimization problems depending on a parameter. We show that, if constraint regularity, a second-order sufficient optimality condition, and a stability condition for the Lagrange multipliers hold, then for sufficiently smooth perturbations of the constraints and the objective function the optimal solutions locally obey a type of Lipschitz condition. The results are applied to finite-dimensional problems, equality constrained problems, and optimal control problems.     

An Extension of Yuan’s Lemma and Its Applications in Optimization

We prove an extension of Yuan’s lemma to more than two matrices, as long as the set of matrices has rank at most 2. This is used to generalize the main result of Baccari and Trad (SIAM J Optim 15(2):394–408, 2005), where the classical necessary second-order optimality condition is proved, under the assumption that the set of Lagrange multipliers is a bounded line segment. We prove the result under the more general assumption that the Hessian of the Lagrangian, evaluated at the vertices of the Lagrange multiplier set, is a matrix set with at most rank 2. We apply the results to prove the classical second-order optimality condition to problems with quadratic constraints and without constant rank of the Jacobian matrix.     

Some kind of Pareto stationarity for multiobjective problems with equilibrium constraints

ABSTRACT

In this paper, we derive some optimality and stationarity conditions for a multiobjective problem with equilibrium constraints (MOPEC). In particular, under a generalized Guignard constraint qualification, we show that any locally Pareto optimal solution of MOPEC must satisfy the strong Pareto Kuhn-Tucker optimality conditions. We also prove that the generalized Guignard constraint qualification is the weakest constraint qualification for the strong Pareto Kuhn-Tucker optimality. Furthermore, under certain convexity or generalized convexity assumptions, we show that the strong Pareto Kuhn-Tucker optimality conditions are also sufficient for several popular locally Pareto-type optimality conditions for MOPEC.