The semismooth Newton method for the solution of quasi-variational inequalities

We consider the application of the globalized semismooth Newton method to the solution of (the KKT conditions of) quasi variational inequalities. We show that the method is globally and locally superlinearly convergent for some important classes of quasi variational inequality problems. We report numerical results to illustrate the practical behavior of the method.     

Karush-Kuhn-Tucker systems: regularity conditions,error bounds and a class of Newton-type methods

 We consider optimality systems of Karush-Kuhn-Tucker (KKT) type, which arise, for example, as primal-dual conditions characterizing solutions of optimization problems or variational inequalities. In particular, we discuss error bounds and Newton-type methods for such systems. An exhaustive comparison of various regularity conditions which arise in this context is given. We obtain a new error bound under an assumption which we show to be strictly weaker than assumptions previously used for KKT systems, such as quasi-regularity or semistability (equivalently, the R ₀-property). Error bounds are useful, among other things, for identifying active constraints and developing efficient local algorithms. We propose a family of local Newton-type algorithms. This family contains some known active-set Newton methods, as well as some new methods. Regularity conditions required for local superlinear convergence compare favorably with convergence conditions of nonsmooth Newton methods and sequential quadratic programming methods. Received: December 10, 2001 / Accepted: July 28, 2002 Published online: February 14, 2003 Key words. KKT system – regularity – error bound – active constraints – Newton method Mathematics Subject Classification (1991): 90C30, 65K05     

一类广义半无限规划问题的光滑牛顿算法

本文在广义半无限规划问题的最优解集X处满足某些条件的前提下将广义半无限规划问题转化成KKT系统,通过扰动的FB函数,将KKT系统转化为一组光滑函数方程,设计了一个光滑牛顿算法,证明了算法的全局收敛性,并且在光滑函数解集处满足局部误差界条件下证明了算法具有超线性收敛速率.     

Structured regularization for barrier NLP solvers

Barrier methods have led to several nonlinear programming (NLP) solvers (e.g. IPOPT, KNITRO, LOQO). However, certain regularity conditions are required for convergence of these methods. These conditions are violated for optimization models with dependent constraints, thus leading to method failure. These shortcomings can be identified by checking the inertia of the KKT matrix, and current solvers either add regularizing terms to correct the inertia of the KKT matrix or revert to more expensive trust region methods to solve the barrier problem. This study improves on these approaches with a new structured regularization strategy; within the Newton step it identifies an independent subset of equality constraints and removes the remaining constraints without modifying the KKT matrix structure. This approach leads to more accurate Newton steps and faster convergence, while maintaining global convergence properties. Implemented in IPOPT with linear solvers HSL_MA57, HSL_MA97 and MUMPS, we present numerical experiments on hundreds of examples from the CUTEr test set, modified for dependency. These results show an average reduction in iterations of more than 50 % over the current version of IPOPT. In addition, several nonlinear blending problems are solved with the proposed algorithm, and improvements over existing regularization strategies are further demonstrated.     

3-分片线性NCP函数的滤子QP-free算法

本文定义一个3-分片线性的NCP函数,并对非线性约束优化问题,提出了带有这分片NCP函数的QP-free非可行域算法.根据优化问题的一阶KKT条件,利用乘子和NCP函数,得到非光滑方程,本文给出一个非光滑方程的迭代算法.这算法包含原始-对偶变量,在局部意义下,可看成关于一阶KKT最优条件的的扰动拟牛顿迭代算法.在线性搜索时,这算法采用滤子方法.本文给出的算法是可实现的并具有全局收敛性,且在适当假设下具有超线性收敛性.     

分片线性NCP函数滤子QP-free算法

本文定义了分片线性NCP函数,并对非线性约束优化问题,提出了带有这分片NCP函数的QP-free非可行域算法.利用优化问题的一阶KKT条件,乘子和NCP函数,得到对应的非光滑方程组.本文给出解这非光滑方程组算法,它包含原始-对偶变量,在局部意义下,可看成关扰动牛顿-拟牛顿迭代算法.在线性搜索时,这算法采用滤子方法.本文给出的算法是可实现的并具有全局收敛性,在适当假设下算法具有超线性收敛性.     

新的滤子方法

本文定义了一种新的滤子方法,并提出了求解光滑不等式约束最优化问题的滤子QP-free非可行域方法.通过乘子和分片线性非线性互补函数,构造一个等价于原约束问题一阶KKT条件的非光滑方程组.在此基础上,通过牛顿-拟牛顿迭代得到满足KKT最优条件的解,在迭代中采用了滤子线搜索方法,证明了该算法是可实现,并具有全局收敛性.另外,在较弱条件下可以证明该方法具有超线性收敛性.     

Local path-following property of inexact interior methods in nonlinear programming

We study the local behavior of a primal-dual inexact interior point methods for solving nonlinear systems arising from the solution of nonlinear optimization problems or more generally from nonlinear complementarity problems. The algorithm is based on the Newton method applied to a sequence of perturbed systems that follows by perturbation of the complementarity equations of the original system. In case of an exact solution of the Newton system, it has been shown that the sequence of iterates is asymptotically tangent to the central path (Armand and Benoist in Math. Program. 115:199?C222, 2008). The purpose of the present paper is to extend this result to an inexact solution of the Newton system. We give quite general conditions on the different parameters of the algorithm, so that this asymptotic property is satisfied. Some numerical tests are reported to illustrate our theoretical results.     

一类二次半定规划内点算法的搜索方向

利用牛顿法求解一类二次半定规划的扰动KKT方程组,得出这类二次半定规划原始-对偶路径跟踪算法搜索方向求解的统一形式,以及HKM搜索方向和NT搜索方向存在唯一的充分条件,最后给出了计算搜索方向的表达式,和特殊情况下搜索方向的计算方法.     

Nonsingularity of FB system and constraint nondegeneracy in semidefinite programming

For a KKT point of the linear semidefinite programming (SDP), we show that the nonsingularity of the B-subdifferential of Fischer-Burmeister (FB) nonsmooth system, the nonsingularity of Clarke’s Jacobian of this system, and the primal and dual constraint nondegeneracies, are all equivalent. Also, each of these conditions is equivalent to the nonsingularity of Clarke’s Jacobian of the smoothed counterpart of FB nonsmooth system, which particularly implies that the FB smoothing Newton method may attain the local quadratic convergence without strict complementarity assumption. We also report numerical results of the FB smoothing method for some benchmark problems.     

On the Newton Interior-Point Method for Nonlinear Programming Problems

Interior-point methods have been developed largely for nonlinear programming problems. In this paper, we generalize the global Newton interior-point method introduced in Ref. 1 and we establish a global convergence theory for it, under the same assumptions as those stated in Ref. 1. The generalized algorithm gives the possibility of choosing different descent directions for a merit function so that difficulties due to small steplength for the perturbed Newton direction can be avoided. The particular choice of the perturbation enables us to interpret the generalized method as an inexact Newton method. Also, we suggest a more general criterion for backtracking, which is useful when the perturbed Newton system is not solved exactly. We include numerical experimentation on discrete optimal control problems.     

无罚函数和滤子的QP-free非可行域方法

提出了求解光滑不等式约束最优化问题的无罚函数和无滤子QP-free非可行域方法. 通过乘子和非线性互补函数, 构造一个等价于原约束问题一阶KKT条件的非光滑方程组. 在此基础上, 通过牛顿-拟牛顿迭代得到满足KKT最优性条件的解, 在迭代中采用了无罚函数和无滤子线搜索方法, 并证明该算法是可实现,具有全局收敛性. 另外, 在较弱条件下可以证明该方法具有超线性收敛性.     

A nonsmooth Levenberg–Marquardt method for solving semi-infinite programming problems

In this paper, we first transform the semi-infinite programming problem into the KKT system by the techniques in [D.H. Li, L. Qi, J. Tam, S.Y. Wu, A smoothing Newton method for semi-infinite programming, J. Global. Optim. 30 (2004) 169–194; L. Qi, S.Y. Wu, G.L. Zhou, Semismooth Newton methods for solving semi-infinite programming problems, J. Global. Optim. 27 (2003) 215–232]. Then a nonsmooth and inexact Levenberg–Marquardt method is proposed for solving this KKT system based on [H. Dan, N. Yamashita, M. Fukushima, Convergence properties of the inexact Levenberg–Marquardt method under local error bound conditions, Optimim. Methods Softw., 11 (2002) 605–626]. This method is globally and superlinearly (even quadratically) convergent. Finally, some numerical results are given.     

Stability of planar shock fronts for multidimensional systems of relaxation equations

We investigate stability of multidimensional planar shock profiles of a general hyperbolic relaxation system whose equilibrium model is a system, under the necessary assumption of spectral stability and a standard set of structural conditions that are known to hold for many physical systems. Our main result, generalizing the work of Kwon and Zumbrun in the scalar relaxation case, is to establish the bounds on the Green?s function for the linearized equation and obtain nonlinear L² asymptotic behavior/sharp decay rate of perturbed weak shock profiles. To establish Green?s function bounds, we use the semigroup approach in the low-frequency regime, and use the energy method for the high-frequency bounds, separately. For the system equilibrium case, the analysis of the linearized equation is complicated due to glancing phenomena. We treat this difficulty similarly as in the inviscid and viscous systems, under the constant multiplicity condition.     

An augmented Lagrangian interior-point method using directions of negative curvature

 We describe an efficient implementation of an interior-point algorithm for non-convex problems that uses directions of negative curvature. These directions should ensure convergence to second-order KKT points and improve the computational efficiency of the procedure. Some relevant aspects of the implementation are the strategy to combine a direction of negative curvature and a modified Newton direction, and the conditions to ensure feasibility of the iterates with respect to the simple bounds. The use of multivariate barrier and penalty parameters is also discussed, as well as the update rules for these parameters. We analyze the convergence of the procedure; both the linesearch and the update rule for the barrier parameter behave appropriately. As the main goal of the paper is the practical usage of negative curvature, a set of numerical results on small test problems is presented. Based on these results, the relevance of using directions of negative curvature is discussed. Received: July 2000 / Accepted: October 2002 Published online: December 19, 2002 Key words. Primal-dual methods – Nonconvex optimization – Linesearches Research supported by Spanish MEC grant BEC2000-0167 Mathematics Subject Classification (1991): 49M37, 65K05, 90C30     

A QP Free Feasible Method

In [12], a QP free feasible method was proposed for the minimization of a smooth function subject to smooth inequality constraints. This method is based on the solutions of linear systems of equations, the reformulation of the KKT optimality conditions by using the Fischer-Burmeister NCP function. This method ensures the feasibility of all iterations. In this paper, we modify the method in [12] slightly to obtain the local convergence under some weaker conditions. In particular, this method is implementable and globally convergent without assuming the linear independence of the gradients of active constrained functions and the uniformly positive definiteness of the submatrix obtained by the Newton or Quasi Newton methods. We also prove that the method has superlinear convergence rate under some mild conditions. Some preliminary numerical results indicate that this new QP free feasible method is quite promising.     

Strong calmness of perturbed KKT system for a class of conic programming with degenerate solutions

ABSTRACT

This paper is concerned with the strong calmness of the KKT solution mapping for a class of canonically perturbed conic programming, which plays a central role in achieving fast convergence under situations when the Lagrange multiplier associated to a solution of these conic optimization problems is not unique. We show that the strong calmness of the KKT solution mapping is equivalent to a local error bound for the solutions to the perturbed KKT system, and is also equivalent to the pseudo-isolated calmness of the stationary point mapping along with the calmness of the multiplier set mapping at the corresponding reference point. Sufficient conditions are also provided for the strong calmness by establishing the pseudo-isolated calmness of the stationary point mapping in terms of the noncriticality of the associated multiplier, and the calmness of the multiplier set mapping in terms of a relative interior condition for the multiplier set. These results cover and extend the existing ones in Hager and Gowda [Stability in the presence of degeneracy and error estimation. Math Program. 1999;85:181–192]; Izmailov and Solodov [Stabilized SQP revisited. Math Program. 2012;133:93–120] for nonlinear programming and in Cui et al. [On the asymptotic superlinear convergence of the augmented Lagrangian method for semidefinite programming with multiple solutions. 2016, arXiv: 1610.00875v1]; Zhang and Zhang [Critical multipliers in semidefinite programming. 2018, arXiv: 1801.02218v1] for semidefinite programming.     

A Gauss–Newton Approach for Solving Constrained Optimization Problems Using Differentiable Exact Penalties

We propose a Gauss–Newton-type method for nonlinear constrained optimization using the exact penalty introduced recently by André and Silva for variational inequalities. We extend their penalty function to both equality and inequality constraints using a weak regularity assumption, and as a result, we obtain a continuously differentiable exact penalty function and a new reformulation of the KKT conditions as a system of equations. Such reformulation allows the use of a semismooth Newton method, so that local superlinear convergence rate can be proved under an assumption weaker than the usual strong second-order sufficient condition and without requiring strict complementarity. Besides, we note that the exact penalty function can be used to globalize the method. We conclude with some numerical experiments using the collection of test problems CUTE.     

非线性规划的QP-free方法

该文提出一种QP-free可行域方法用来解满足光滑不等式约束的最优化问题.此方法把QP-free方法和3-1线性互补函数相结合一个等价于原约束问题的一阶KKT条件的方程组,并在此基础上给出解这个方程组的迭代算法. 这个方法的每一步迭代都可以看作是对求KKT条件解的牛顿或拟牛顿迭代的扰动,且在该方法中每一步的迭代均具有可行性. 该方法是可实行的且具有全局性, 且不需要严格互补条件、聚点的孤立性和积极约束函数梯度的线性独立等假设. 在与文献[2]中相同的适当条件下,此方法还具有超线性收敛性. 数值检验结果表示,该文提出的QP-free可行域方法是切实有效的方法.     

Exact penalties for variational inequalities with applications to nonlinear complementarity problems

In this paper, we present a new reformulation of the KKT system associated to a variational inequality as a semismooth equation. The reformulation is derived from the concept of differentiable exact penalties for nonlinear programming. The best theoretical results are presented for nonlinear complementarity problems, where simple, verifiable, conditions ensure that the penalty is exact. We close the paper with some preliminary computational tests on the use of a semismooth Newton method to solve the equation derived from the new reformulation. We also compare its performance with the Newton method applied to classical reformulations based on the Fischer-Burmeister function and on the minimum. The new reformulation combines the best features of the classical ones, being as easy to solve as the reformulation that uses the Fischer-Burmeister function while requiring as few Newton steps as the one that is based on the minimum.