Stabilized SQP revisited

The stabilized version of the sequential quadratic programming algorithm (sSQP) had been developed in order to achieve superlinear convergence in situations when the Lagrange multipliers associated to a solution are not unique. Within the framework of Fischer (Math Program 94:91–124, 2002), the key to local superlinear convergence of sSQP are the following two properties: upper Lipschitzian behavior of solutions of the Karush-Kuhn-Tucker (KKT) system under canonical perturbations and local solvability of sSQP subproblems with the associated primal-dual step being of the order of the distance from the current iterate to the solution set of the unperturbed KKT system. According to Fernández and Solodov (Math Program 125:47–73, 2010), both of these properties are ensured by the second-order sufficient optimality condition (SOSC) without any constraint qualification assumptions. In this paper, we state precise relationships between the upper Lipschitzian property of solutions of KKT systems, error bounds for KKT systems, the notion of critical Lagrange multipliers (a subclass of multipliers that violate SOSC in a very special way), the second-order necessary condition for optimality, and solvability of sSQP subproblems. Moreover, for the problem with equality constraints only, we prove superlinear convergence of sSQP under the assumption that the dual starting point is close to a noncritical multiplier. Since noncritical multipliers include all those satisfying SOSC but are not limited to them, we believe this gives the first superlinear convergence result for any Newtonian method for constrained optimization under assumptions that do not include any constraint qualifications and are weaker than SOSC. In the general case when inequality constraints are present, we show that such a relaxation of assumptions is not possible. We also consider applying sSQP to the problem where inequality constraints are reformulated into equalities using slack variables, and discuss the assumptions needed for convergence in this approach. We conclude with consequences for local regularization methods proposed in (Izmailov and Solodov SIAM J Optim 16:210–228, 2004; Wright SIAM J. Optim. 15:673–676, 2005). In particular, we show that these methods are still locally superlinearly convergent under the noncritical multiplier assumption, weaker than SOSC employed originally.     

On attraction of Newton-type iterates to multipliers violating second-order sufficiency conditions

Assuming that the primal part of the sequence generated by a Newton-type (e.g., SQP) method applied to an equality-constrained problem converges to a solution where the constraints are degenerate, we investigate whether the dual part of the sequence is attracted by those Lagrange multipliers which satisfy second-order sufficient condition (SOSC) for optimality, or by those multipliers which violate it. This question is relevant at least for two reasons: one is speed of convergence of standard methods; the other is applicability of some recently proposed approaches for handling degenerate constraints. We show that for the class of damped Newton methods, convergence of the dual sequence to multipliers satisfying SOSC is unlikely to occur. We support our findings by numerical experiments. We also suggest a simple auxiliary procedure for computing multiplier estimates, which does not have this undesirable property. Finally, some consequences for the case of mixed equality and inequality constraints are discussed.     

Trust-region problems with linear inequality constraints: exact SDP relaxation,global optimality and robust optimization

The trust-region problem, which minimizes a nonconvex quadratic function over a ball, is a key subproblem in trust-region methods for solving nonlinear optimization problems. It enjoys many attractive properties such as an exact semi-definite linear programming relaxation (SDP-relaxation) and strong duality. Unfortunately, such properties do not, in general, hold for an extended trust-region problem having extra linear constraints. This paper shows that two useful and powerful features of the classical trust-region problem continue to hold for an extended trust-region problem with linear inequality constraints under a new dimension condition. First, we establish that the class of extended trust-region problems has an exact SDP-relaxation, which holds without the Slater constraint qualification. This is achieved by proving that a system of quadratic and affine functions involved in the model satisfies a range-convexity whenever the dimension condition is fulfilled. Second, we show that the dimension condition together with the Slater condition ensures that a set of combined first and second-order Lagrange multiplier conditions is necessary and sufficient for global optimality of the extended trust-region problem and consequently for strong duality. Through simple examples we also provide an insightful account of our development from SDP-relaxation to strong duality. Finally, we show that the dimension condition is easily satisfied for the extended trust-region model that arises from the reformulation of a robust least squares problem (LSP) as well as a robust second order cone programming model problem (SOCP) as an equivalent semi-definite linear programming problem. This leads us to conclude that, under mild assumptions, solving a robust LSP or SOCP under matrix-norm uncertainty or polyhedral uncertainty is equivalent to solving a semi-definite linear programming problem and so, their solutions can be validated in polynomial time.     

Second-Order Modified Contingent Epiderivatives in Set-Valued Optimization

In this article, we introduce a second-order modified contingent cone and a second-order modified contingent epiderivative. We discuss some properties of the second-order cone and the epiderivative, respectively. Moreover, a Fritz John type necessary optimality condition is obtained for the set-valued optimization problems with constraints by using the second-order modified contingent epiderivative and an example is proposed to explain the Fritz John type necessary optimality condition. In particular, we obtain a unified second-order sufficient and necessary optimality condition for the set-valued optimization problems with constraints under twice differentiable L-quasi-convex assumption.     

集值优化问题超有效解的Lagrange最优性条件

在局部凸空间中考虑约束集值优化问题（VP）在超有效解意义下的Lagrange最优性条件．在近似锥-次类凸假设下，利用择一性定理得到了（VP）取得强有效解的必要条件，利用超有效解集的性质及超有效解的定义给出了（VP）取得超有效解的充分条件，最后给出了一种与（VP）等价的无约束规划．     

Stabilized sequential quadratic programming for optimization and a stabilized Newton-type method for variational problems

The stabilized version of the sequential quadratic programming algorithm (sSQP) had been developed in order to achieve fast convergence despite possible degeneracy of constraints of optimization problems, when the Lagrange multipliers associated to a solution are not unique. Superlinear convergence of sSQP had been previously established under the strong second-order sufficient condition for optimality (without any constraint qualification assumptions). We prove a stronger superlinear convergence result than the above, assuming the usual second-order sufficient condition only. In addition, our analysis is carried out in the more general setting of variational problems, for which we introduce a natural extension of sSQP techniques. In the process, we also obtain a new error bound for Karush–Kuhn–Tucker systems for variational problems that holds under an appropriate second-order condition.     

Semidiscrete Ritz-Galerkin approximation of nonlinear parabolic boundary control problems—Strong convergence of optimal controls

A class of optimal control problems for a parabolic equation with nonlinear boundary condition and constraints on the control and the state is considered. Associated approximate problems are established, where the equation of state is defined by a semidiscrete Ritz-Galerkin method. Moreover, we are able to allow for the discretization of admissible controls. We show the convergence of the approximate controls to the solution of the exact control problem, as the discretization parameter tends toward zero. This result holds true under the assumption of a certain sufficient second-order optimality condition.Dedicated to the 60th birthday of Lothar von Wolfersdorf     

Optimization of Mayer Problem with Sturm–Liouville-Type Differential Inclusions

The present paper studies a new class of problems of optimal control theory with Sturm–Liouville-type differential inclusions involving second-order linear self-adjoint differential operators. Our main goal is to derive the optimality conditions of Mayer problem for differential inclusions with initial point constraints. By using the discretization method guaranteeing transition to continuous problem, the discrete and discrete-approximation inclusions are investigated. Necessary and sufficient conditions, containing both the Euler–Lagrange and Hamiltonian-type inclusions and “transversality” conditions are derived. The idea for obtaining optimality conditions of Mayer problem is based on applying locally adjoint mappings. This approach provides several important equivalence results concerning locally adjoint mappings to Sturm–Liouville-type set-valued mappings. The result strengthens and generalizes to the problem with a second-order non-self-adjoint differential operator; a suitable choice of coefficients then transforms this operator to the desired Sturm–Liouville-type problem. In particular, if a positive-valued, scalar function specific to Sturm–Liouville differential inclusions is identically equal to one, we have immediately the optimality conditions for the second-order discrete and differential inclusions. Furthermore, practical applications of these results are demonstrated by optimization of some “linear” optimal control problems for which the Weierstrass–Pontryagin maximum condition is obtained.     

Stability of solutions to control constrained nonlinear optimal control problems

We consider a family of nonlinear optimal control problems depending on a parameter. Under the assumption of a second-order sufficient optimality condition it is shown that the solutions of the problems as well as the associated Lagrange multipliers are Lipschitz continuous functions of the parameter.     

On a Conjecture in Second-Order Optimality Conditions

In this paper, we deal with a conjecture formulated in Andreani et al. (Optimization 56:529–542, 2007), which states that whenever a local minimizer of a nonlinear optimization problem fulfills the Mangasarian–Fromovitz constraint qualification and the rank of the set of gradients of active constraints increases at most by one in a neighborhood of the minimizer, a second-order optimality condition that depends on one single Lagrange multiplier is satisfied. This conjecture generalizes previous results under a constant rank assumption or under a rank deficiency of at most one. We prove the conjecture under the additional assumption that the Jacobian matrix has a smooth singular value decomposition. Our proof also extends to the case of the strong second-order condition, defined in terms of the critical cone instead of the critical subspace.     

Transversality condition and optimization of higher order ordinary differential inclusions

In the present paper, a Bolza problem of optimal control theory with a fixed time interval given by convex and nonconvex second-order differential inclusions (P_H) is studied. Our main goal is to derive sufficient optimality conditions for Cauchy problem of sth-order differential inclusions. The sufficient conditions including distinctive transversality condition are proved incorporating the Euler–Lagrange and Hamiltonian type inclusions. The basic concepts involved in obtaining optimality conditions are the locally adjoint mappings. Furthermore, the application of these results is demonstrated by solving the problems with third-order differential inclusions.     

Mathematical Programs with Vanishing Constraints: Optimality Conditions,Sensitivity, and a Relaxation Method

We consider a class of optimization problems with switch-off/switch-on constraints, which is a relatively new problem model. The specificity of this model is that it contains constraints that are being imposed (switched on) at some points of the feasible region, while being disregarded (switched off) at other points. This seems to be a potentially useful modeling paradigm, that has been shown to be helpful, for example, in optimal topology design. The fact that some constraints “vanish” from the problem at certain points, gave rise to the name of mathematical programs with vanishing constraints (MPVC). It turns out that such problems are usually degenerate at a solution, but are structurally different from the related class of mathematical programs with complementarity constraints (MPCC). In this paper, we first discuss some known first- and second-order necessary optimality conditions for MPVC, giving new very short and direct justifications. We then derive some new special second-order sufficient optimality conditions for these problems and show that, quite remarkably, these conditions are actually equivalent to the classical/standard second-order sufficient conditions in optimization. We also provide a sensitivity analysis for MPVC. Finally, a relaxation method is proposed. For this method, we analyze constraints regularity and boundedness of the Lagrange multipliers in the relaxed subproblems, derive a sufficient condition for local uniqueness of solutions of subproblems, and give convergence estimates. Research of the first author was supported by the Russian Foundation for Basic Research Grants 07-01-00270, 07-01-00416 and 07-01-90102-Mong, and by RF President’s Grant NS-9344.2006.1 for the support of leading scientific schools. The second author was supported in part by CNPq Grants 301508/2005-4, 490200/2005-2 and 550317/2005-8, by PRONEX-Optimization, and by FAPERJ.     

Study of a special nonlinear problem arising in convex semi-infinite programming

We consider convex problems of semi-infinite programming (SIP) using an approach based on the implicit optimality criterion. This criterion allows one to replace optimality conditions for a feasible solution x ⁰ of the convex SIP problem by such conditions for x ⁰ in some nonlinear programming (NLP) problem denoted by NLP(I(x ⁰)). This nonlinear problem, constructed on the base of special characteristics of the original SIP problem, so-called immobile indices and their immobility orders, has a special structure and a diversity of important properties. We study these properties and use them to obtain efficient explicit optimality conditions for the problem NLP(I(x ⁰)). Application of these conditions, together with the implicit optimality criterion, gives new efficient optimality conditions for convex SIP problems. Special attention is paid to SIP problems whose constraints do not satisfy the Slater condition and to problems with analytic constraint functions for which we obtain optimality conditions in the form of a criterion. Comparison with some known optimality conditions for convex SIP is provided.     

On the No-Gap Second-Order Optimality Conditions for a Discrete Optimal Control Problem with Mixed Constraints

Motivated by our recent works on optimality conditions in discrete optimal control problems under a nonconvex cost function, in this paper, we study second-order necessary and sufficient optimality conditions for a discrete optimal control problem with a nonconvex cost function and state-control constraints. By establishing an abstract result on second-order optimality conditions for a mathematical programming problem, we derive second-order necessary and sufficient optimality conditions for a discrete optimal control problem. Using a common critical cone for both the second-order necessary and sufficient optimality conditions, we obtain “no-gap” between second-order optimality conditions.     

Characterizations of optimal solution sets of convex infinite programs

In this paper, several Lagrange multiplier characterizations of the solution set of a convex infinite programming problem are given. Characterizations of solution sets of cone-constrained convex programs are derived as well. The procedure is then adopted to a semi-convex problem with convex constraints. For this problem, we present firstly a necessary and sufficient condition for optimality and secondly a characterization of its optimal solution set, based on a Lagrange multiplier associated with a given solution and on directional derivatives of the objective function.      

Global Quadratic Minimization over Bivalent Constraints: Necessary and Sufficient Global Optimality Condition

In this paper, we establish global optimality conditions for quadratic optimization problems with quadratic equality and bivalent constraints. We first present a necessary and sufficient condition for a global minimizer of quadratic optimization problems with quadratic equality and bivalent constraints. Then we examine situations where this optimality condition is equivalent to checking the positive semidefiniteness of a related matrix, and so, can be verified in polynomial time by using elementary eigenvalues decomposition techniques. As a consequence, we also present simple sufficient global optimality conditions, which can be verified by solving a linear matrix inequality problem, extending several known sufficient optimality conditions in the existing literature.     

Stability of solutions for a class of nonlinear cone constrained optimization problems,part 1: Basic theory

We Gonsider a class of nonlinear cone constrained optimization problems depending on a parameter. Under the assumption of a constraint qualification, a second order sufficient optimality condition and a stability condition for the Lagrange multipliers it is shown, that for sufficiently smooth perturbations of the constraints and the objective function the optimal solutions obey a type of Lipschitz condition.     

Local stability of solutions to differentiable optimization problems in banach spaces

This paper considers a class of nonlinear differentiable optimization problems depending on a parameter. We show that, if constraint regularity, a second-order sufficient optimality condition, and a stability condition for the Lagrange multipliers hold, then for sufficiently smooth perturbations of the constraints and the objective function the optimal solutions locally obey a type of Lipschitz condition. The results are applied to finite-dimensional problems, equality constrained problems, and optimal control problems.     

An Extension of Yuan’s Lemma and Its Applications in Optimization

We prove an extension of Yuan’s lemma to more than two matrices, as long as the set of matrices has rank at most 2. This is used to generalize the main result of Baccari and Trad (SIAM J Optim 15(2):394–408, 2005), where the classical necessary second-order optimality condition is proved, under the assumption that the set of Lagrange multipliers is a bounded line segment. We prove the result under the more general assumption that the Hessian of the Lagrangian, evaluated at the vertices of the Lagrange multiplier set, is a matrix set with at most rank 2. We apply the results to prove the classical second-order optimality condition to problems with quadratic constraints and without constant rank of the Jacobian matrix.     

Second-order Karush–Kuhn–Tucker optimality conditions for set-valued optimization

In this paper, we propose the concept of a second-order composed contingent derivative for set-valued maps, discuss its relationship to the second-order contingent derivative and investigate some of its special properties. By virtue of the second-order composed contingent derivative, we extend the well-known Lagrange multiplier rule and the Kurcyusz–Robinson–Zowe regularity assumption to a constrained set-valued optimization problem in the second-order case. Simultaneously, we also establish some second-order Karush–Kuhn–Tucker necessary and sufficient optimality conditions for a set-valued optimization problem, whose feasible set is determined by a set-valued map, under a generalized second-order Kurcyusz–Robinson–Zowe regularity assumption.