Mathematical programs with vanishing constraints: constraint qualifications,their applications,and a new regularization method

ABSTRACT

We propose a new family of relaxation schemes for mathematical programs with vanishing constraints that extend the relaxation of Hoheisel, Kanzow & Schwartz from 2012. We discuss the properties of the sequence of relaxed non-linear programs as well as stationary properties of limiting points. Our relaxation schemes have the desired property of converging to an M-stationary point. A main advantage of this new method is to converge to an S-stationary point satisfying MPVC-LICQ for a large class of problem. We also study MPVC constraint qualification connected to this study and prove convergence of the method under the new MPVC-CRSC. Additionally, we obtain the new MPVC-wGCQ and prove that it is the weakest MPVC constraint qualification.     

On the convergence of general regularization and smoothing schemes for mathematical programs with complementarity constraints

We extend the convergence analysis of a smoothing method [M. Fukushima and J.-S. Pang (2000). Convergence of a smoothing continuation method for mathematical programs with complementarity constraints. In: M. Théra and R. Tichatschke (Eds.), Ill-posed Variational Problems and Regularization Techniques, pp. 99–110. Springer, Berlin/Heidelberg.] to a general class of smoothing functions and show that a weak second-order necessary optimality condition holds at the limit point of a sequence of stationary points found by the smoothing method. We also show that convergence and stability results in [S. Scholtes (2001). Convergence properties of a regularization scheme for mathematical programs with complementarity constraints. SIAM J. Optim., 11, 918–936.] hold for a relaxation problem suggested by Scholtes [S. Scholtes (2003). Private communications.] using a class of smoothing functions. In addition, the relationship between two technical, yet critical, concepts in [M. Fukushima and J.-S. Pang (2000). Convergence of a smoothing continuation method for mathematical programs with complementarity constraints. In: M. Théra and R. Tichatschke (Eds.), Ill-posed Variational Problems and Regularization Techniques, pp. 99–110. Springer, Berlin/Heidelberg; S. Scholtes (2001). Convergence properties of a regularization scheme for mathematical programs with complementarity constraints. SIAM J. Optim., 11, 918–936.] for the convergence analysis of the smoothing and regularization methods is discussed and a counter-example is provided to show that the stability result in [S. Scholtes (2001). Convergence properties of a regularization scheme for mathematical programs with complementarity constraints. SIAM J. Optim., 11, 918–936.] cannot be extended to a weaker regularization.     

A log-exponential smoothing method for mathematical programs with complementarity constraints

In this paper a log-exponential smoothing method for mathematical programs with complementarity constraints (MPCC) is analyzed, with some new interesting properties and convergence results provided. It is shown that the stationary points of the resulting smoothed problem converge to the strongly stationary point of MPCC, under the linear independence constraint qualification (LICQ), the weak second-order necessary condition (WSONC), and some reasonable assumption. Moreover, the limit point satisfies the weak second-order necessary condition for MPCC. A notable fact is that the proposed convergence results do not restrict the complementarity constraint functions approach to zero at the same order of magnitude.     

关于求解带线性互补约束的数学规划问题正则方法的一个注记

本文主要研究在某些较弱条件下求解带线性互补约束的数学规划问题(MPLCC)正则方法的收敛性.若衡约束规划线性独立约束规范条件(MPEC-LICQ)在由正则方法产生的点列的聚点处成立,且迭代点列满足二阶必要条件,同时,若比在文[7]中渐近弱非退化条件Ⅰ更弱的渐近弱非退化条件Ⅱ在聚点处也成立,那么所有这些聚点都是B-稳定点.此外,在弱MPEC-LICQ,二阶必要条件及渐近弱退化条件Ⅱ下,我们仍能证明通过正则方法所得的聚点都是B-稳定点.     

Partial Augmented Lagrangian Method and Mathematical Programs with Complementarity Constraints

In this paper, we apply a partial augmented Lagrangian method to mathematical programs with complementarity constraints (MPCC). Specifically, only the complementarity constraints are incorporated into the objective function of the augmented Lagrangian problem while the other constraints of the original MPCC are retained as constraints in the augmented Lagrangian problem. We show that the limit point of a sequence of points that satisfy second-order necessary conditions of the partial augmented Lagrangian problems is a strongly stationary point (hence a B-stationary point) of the original MPCC if the limit point is feasible to MPCC, the linear independence constraint qualification for MPCC and the upper level strict complementarity condition hold at the limit point. Furthermore, this limit point also satisfies a second-order necessary optimality condition of MPCC. Numerical experiments are done to test the computational performances of several methods for MPCC proposed in the literature. This research was partially supported by the Research Grants Council (BQ654) of Hong Kong and the Postdoctoral Fellowship of The Hong Kong Polytechnic University. Dedicated to Alex Rubinov on the occassion of his 65th birthday.     

Global Convergence of a Smooth Approximation Method for Mathematical Programs with Complementarity Constraints

A new smoothing approach was given for solving the mathematical programs with complementarity constraints (MPCC) by using the aggregation technique. As the smoothing parameter tends to zero, if the KKT point sequence generated from the smoothed problems satisfies the second-order necessary condition, then any accumulation point of the sequence is a B-stationary point of MPCC if the linear independence constraint qualification (LICQ) and the upper level strict complementarity (ULSC) condition hold at the limit point. The ULSC condition is weaker than the lower level strict complementarity (LLSC) condition generally used in the literatures. Moreover, the method can be easily extended to the mathematical programs with general vertical complementarity constraints.     

Parametric dual regularization for an optimal control problem with pointwise state constraints

The perturbation method is used in the dual regularization theory for a linear convex optimal control problem with a strongly convex objective functional and pointwise state constraints understood as ones in L ₂. Primary attention is given to the qualitative properties of the dual regularization method, depending on the differential properties of the value function (S-function) in the optimization problem. It is shown that the convergence of the method is closely related to the Lagrange principle and the Pontryagin maximum principle. The dual regularization scheme is shown to provide a new method for proving the maximum principle in the problem with pointwise state constraints understood in L ₂ or C. The regularized Lagrange principle in nondifferential form and the regularized Pontryagin maximum principle are discussed. Illustrative examples are presented.     

A new smoothing and regularization Newton method for the symmetric cone complementarity problem

Based on a regularized Chen–Harker–Kanzow–Smale (CHKS) smoothing function, we propose a new smoothing and regularization Newton method for solving the symmetric cone complementarity problem. By using the theory of Euclidean Jordan algebras, we establish the global and local quadratic convergence of the method on certain assumptions. The proposed method uses a nonmonotone line search technique which includes the usual monotone line search as a special case. In addition, our method treats both the smoothing parameter \(\mu \) and the regularization parameter \(\varepsilon \) as independent variables. Preliminary numerical results are reported which indicate that the proposed method is effective.     

Convergence of a Penalty Method for Mathematical Programming with Complementarity Constraints

We adapt the convergence analysis of the smoothing (Ref. 1) and regularization (Ref. 2) methods to a penalty framework for mathematical programs with complementarity constraints (MPCC); we show that the penalty framework shares convergence properties similar to those of these methods. Moreover, we give sufficient conditions for a sequence generated by the penalty framework to be attracted to a B-stationary point of the MPCC.     

Convergence of a Scholtes-type regularization method for cardinality-constrained optimization problems with an application in sparse robust portfolio optimization

We consider general nonlinear programming problems with cardinality constraints. By relaxing the binary variables which appear in the natural mixed-integer programming formulation, we obtain an almost equivalent nonlinear programming problem, which is thus still difficult to solve. Therefore, we apply a Scholtes-type regularization method to obtain a sequence of easier to solve problems and investigate the convergence of the obtained KKT points. We show that such a sequence converges to an S-stationary point, which corresponds to a local minimizer of the original problem under the assumption of convexity. Additionally, we consider portfolio optimization problems where we minimize a risk measure under a cardinality constraint on the portfolio. Various risk measures are considered, in particular Value-at-Risk and Conditional Value-at-Risk under normal distribution of returns and their robust counterparts under moment conditions. For these investment problems formulated as nonlinear programming problems with cardinality constraints we perform a numerical study on a large number of simulated instances taken from the literature and illuminate the computational performance of the Scholtes-type regularization method in comparison to other considered solution approaches: a mixed-integer solver, a direct continuous reformulation solver and the Kanzow–Schwartz regularization method, which has already been applied to Markowitz portfolio problems.     

Generalized stationary points and an interior-point method for mathematical programs with equilibrium constraints

Generalized stationary points of the mathematical program with equilibrium constraints (MPEC) are studied to better describe the limit points produced by interior point methods for MPEC. A primal-dual interior-point method is then proposed, which solves a sequence of relaxed barrier problems derived from MPEC. Global convergence results are deduced under fairly general conditions other than strict complementarity or the linear independence constraint qualification for MPEC (MPEC-LICQ). It is shown that every limit point of the generated sequence is a strong stationary point of MPEC if the penalty parameter of the merit function is bounded. Otherwise, a point with certain stationarity can be obtained. Preliminary numerical results are reported, which include a case analyzed by Leyffer for which the penalty interior-point algorithm failed to find a stationary point.Mathematics Subject Classification (1991):90C30, 90C33, 90C55, 49M37, 65K10     

Tikhonov Regularization Methods for Variational Inequality Problems

Motivated by the work of Facchinei and Kanzow (Ref. 1) on regularization methods for the nonlinear complementarity problem and the work of Ravindran and Gowda (Ref. 2) for the box variational inequality problem, we study regularization methods for the general variational inequality problem. A sufficient condition is given which guarantees that the union of the solution sets of the regularized problems is nonempty and bounded. It is shown that solutions of the regularized problems form a minimizing sequence of the D-gap function under a mild condition.     

A note on the sample average approximation method for stochastic mathematical programs with complementarity constraints

Meng and Xu (2006) [3] proposed a sample average approximation (SAA) method for solving a class of stochastic mathematical programs with complementarity constraints (SMPCCs). After showing that under some moderate conditions, a sequence of weak stationary points of SAA problems converge to a weak stationary point of the original SMPCC with probability approaching one at exponential rate as the sample size tends to infinity, the authors proposed an open question, that is, whether similar results can be obtained under some relatively weaker conditions. In this paper, we try to answer the open question. Based on the reformulation of stationary condition of MPCCs and new stability results on generalized equations, we present a similar convergence theory without any information of second order derivative and strict complementarity conditions. Moreover, we carry out convergence analysis of the regularized SAA method proposed by Meng and Xu (2006) [3] where the convergence results have not been considered.     

Convergence properties of the inexact Lin-Fukushima relaxation method for mathematical programs with complementarity constraints

Mathematical programs with equilibrium (or complementarity) constraints, MPECs for short, form a difficult class of optimization problems. The feasible set of MPECs is described by standard equality and inequality constraints as well as additional complementarity constraints that are used to model equilibrium conditions in different applications. But these complementarity constraints imply that MPECs violate most of the standard constraint qualifications. Therefore, more specialized algorithms are typically applied to MPECs that take into account the particular structure of the complementarity constraints. One popular class of these specialized algorithms are the relaxation (or regularization) methods. They replace the MPEC by a sequence of nonlinear programs NLP(t) depending on a parameter t, then compute a KKT-point of each NLP(t), and try to get a suitable stationary point of the original MPEC in the limit t→0. For most relaxation methods, one can show that a C-stationary point is obtained in this way, a few others even get M-stationary points, which is a stronger property. So far, however, these results have been obtained under the assumption that one is able to compute exact KKT-points of each NLP(t). But this assumption is not implementable, hence a natural question is: What kind of stationarity do we get if we only compute approximate KKT-points? It turns out that most relaxation methods only get a weakly stationary point under this assumption, while in this paper, we show that the smooth relaxation method by Lin and Fukushima (Ann. Oper. Res. 133:63–84, 2005) still yields a C-stationary point, i.e. the inexact version of this relaxation scheme has the same convergence properties as the exact counterpart.     

A Class of Quadratic Programs with Linear Complementarity Constraints

We consider a class of quadratic programs with linear complementarity constraints (QPLCC) which belong to mathematical programs with equilibrium constraints (MPEC). We investigate various stationary conditions and present new and strong necessary and sufficient conditions for global and local optimality. Furthermore, we propose a Newton-like method to find an M-stationary point in finite steps without MEPC linear independence constraint qualification. The research of this author is partially supported by NSERC, and Research Grand Council of Hong Kong.     

Merit-Function Piecewise SQP Algorithm for Mathematical Programs with Equilibrium Constraints

We propose a merit-function piecewise SQP algorithm for mathematical programs with equilibrium constraints (MPEC) formulated as mathematical programs with complementarity constraints. Under mild conditions, the new algorithm is globally convergent to a piecewise stationary point. Moreover, if the partial MPEC linear independence constraint qualification (LICQ) is satisfied at the accumulation point, then the accumulation point is an S-stationary point. The research of the first author was supported by the National Natural Science Foundation of China under grants 10571177 and 70271014. The research of the second author was partially supported by NSERC.     

Lifting mathematical programs with complementarity constraints

We present a new smoothing approach for mathematical programs with complementarity constraints, based on the orthogonal projection of a smooth manifold. We study regularity of the lifted feasible set and, since the corresponding optimality conditions are inherently degenerate, introduce a regularization approach involving a novel concept of tilting stability. A correspondence between the C-index in the original problem and the quadratic index in the lifted problem is shown. In particular, a local minimizer of the mathematical program with complementarity constraints may numerically be found by minimization of the lifted, smooth problem. We report preliminary computational experience with the lifting approach.     

New Relaxation Method for Mathematical Programs with Complementarity Constraints

In this paper, we present a new relaxation method for mathematical programs with complementarity constraints. Based on the fact that a variational inequality problem defined on a simplex can be represented by a finite number of inequalities, we use an expansive simplex instead of the nonnegative orthant involved in the complementarity constraints. We then remove some inequalities and obtain a standard nonlinear program. We show that the linear independence constraint qualification or the Mangasarian–Fromovitz constraint qualification holds for the relaxed problem under some mild conditions. We consider also a limiting behavior of the relaxed problem. We prove that any accumulation point of stationary points of the relaxed problems is a weakly stationary point of the original problem and that, if the function involved in the complementarity constraints does not vanish at this point, it is C-stationary. We obtain also some sufficient conditions of B-stationarity for a feasible point of the original problem. In particular, some conditions described by the eigenvalues of the Hessian matrices of the Lagrangian functions of the relaxed problems are new and can be verified easily. Our limited numerical experience indicates that the proposed approach is promising.     

A class of smoothing SAA methods for a stochastic mathematical program with complementarity constraints

A class of smoothing sample average approximation (SAA) methods is proposed for solving the stochastic mathematical program with complementarity constraints (SMPCC) considered by Birbil et al. [S.I. Birbil, G. Gürkan, O. Listes, Solving stochastic mathematical programs with complementarity constraints using simulation, Math. Oper. Res. 31 (2006) 739–760]. The almost sure convergence of optimal solutions of the smoothed SAA problem to that of the true problem is established by the notion of epi-convergence in variational analysis. It is demonstrated that, under suitable conditions, any accumulation point of Karash–Kuhn–Tucker points of the smoothed SAA problem is almost surely a kind of stationary point of SMPCC as the sample size tends to infinity. Moreover, under a strong second-order sufficient condition for SMPCC, the exponential convergence rate of the sequence of Karash–Kuhn–Tucker points of the smoothed SAA problem is investigated through an application of Robinson?s stability theory. Some preliminary numerical results are reported to show the efficiency of proposed method.     

A Sequential Smooth Penalization Approach to Mathematical Programs with Complementarity Constraints

In this paper, a mathematical program with complementarity constraints (MPCC) is reformulated as a nonsmooth constrained mathematical program via the Fischer–Burmeister function. Smooth penalty functions are used to treat this nonsmooth constrained program. Under linear independence constraint qualification, and upper level strict complementarity condition, together with some other mild conditions, we prove that the limit point of stationary points satisfying second-order necessary conditions of unconstrained penalized problems is a strongly stationary point, hence a B-stationary point of the original MPCC. Furthermore, this limit point also satisfies a second-order necessary condition of the original MPCC. Numerical results are presented to test the performance of this method.