Firefly penalty-based algorithm for bound constrained mixed-integer nonlinear programming

In this article, we aim to extend the firefly algorithm (FA) to solve bound constrained mixed-integer nonlinear programming (MINLP) problems. An exact penalty continuous formulation of the MINLP problem is used. The continuous penalty problem comes out by relaxing the integrality constraints and by adding a penalty term to the objective function that aims to penalize integrality constraint violation. Two penalty terms are proposed, one is based on the hyperbolic tangent function and the other on the inverse hyperbolic sine function. We prove that both penalties can be used to define the continuous penalty problem, in the sense that it is equivalent to the MINLP problem. The solutions of the penalty problem are obtained using a variant of the metaheuristic FA for global optimization. Numerical experiments are given on a set of benchmark problems aiming to analyze the quality of the obtained solutions and the convergence speed. We show that the firefly penalty-based algorithm compares favourably with the penalty algorithm when the deterministic DIRECT or the simulated annealing solvers are invoked, in terms of convergence speed.     

Repulsive Assignment Problem

Standard assignment is the problem of obtaining a matching between two sets of respectively persons and positions so that each person is assigned exactly one position and each position receives exactly one person, while a linear decision maker utility function is maximized. We introduce a variant of the problem where the persons individual utilities are taken into account in a way that a feasible solution must satisfy not only the standard assignment constraints, but also an equilibrium constraint of the complementarity type, which we call repulsive. The equilibrium constraint can be, in turn, transformed into a typically large set of linear constraints. Our problem is NP-hard and it is a special case of the assignment problem with side constraints. We study an exact penalty function approach which motivates a heuristic algorithm. We provide computational experiments that show the usefulness of a heuristic mechanism inspired by the exact approach. The heuristics outperforms a state-of-the-art integer linear programming solver.     

半监督距离度量学习内蕴加速投影梯度算法

考虑求解一类半监督距离度量学习问题. 由于样本集(数据库)的规模与复杂性的激增, 在考虑距离度量学习问题时, 必须考虑学习来的距离度量矩阵具有稀疏性的特点. 因此, 在现有的距离度量学习模型中, 增加了学习矩阵的稀疏约束. 为了便于模型求解, 稀疏约束应用了Frobenius 范数约束. 进一步, 通过罚函数方法将Frobenius范数约束罚到目标函数, 使得具有稀疏约束的模型转化成无约束优化问题. 为了求解问题, 提出了正定矩阵群上加速投影梯度算法, 克服了矩阵群上不能直接进行线性组合的困难, 并分析了算法的收敛性. 最后通过UCI数据库的分类问题的例子, 进行了数值实验, 数值实验的结果说明了学习矩阵的稀疏性以及加速投影梯度算法的有效性.     

Partial Exact Penalty for Mathematical Programs with Equilibrium Constraints

It is well known that mathematical programs with equilibrium constraints (MPEC) violate the standard constraint qualifications such as Mangasarian–Fromovitz constraint qualification (MFCQ) and hence the usual Karush–Kuhn–Tucker conditions cannot be used as stationary conditions unless relatively strong assumptions are satisfied. This observation has led to a number of weaker stationary conditions, with Mordukhovich stationary (M-stationary) condition being the strongest among the weaker conditions. In nonlinear programming, it is known that MFCQ leads to an exact penalization. In this paper we show that MPEC GMFCQ, an MPEC variant of MFCQ, leads to a partial exact penalty where all the constraints except a simple linear complementarity constraint are moved to the objective function. The partial exact penalty function, however, is nonsmooth. By smoothing the partial exact penalty function, we design an algorithm which is shown to be globally convergent to an M-stationary point under an extended version of the MPEC GMFCQ.     

On the Smoothing of the Square-Root Exact Penalty Function for Inequality Constrained Optimization

In this paper we propose two methods for smoothing a nonsmooth square-root exact penalty function for inequality constrained optimization. Error estimations are obtained among the optimal objective function values of the smoothed penalty problem, of the nonsmooth penalty problem and of the original optimization problem. We develop an algorithm for solving the optimization problem based on the smoothed penalty function and prove the convergence of the algorithm. The efficiency of the smoothed penalty function is illustrated with some numerical examples, which show that the algorithm seems efficient.     

Bilevel multiplicative problems: A penalty approach to optimality and a cutting plane based algorithm

Bilevel programming has been proposed for dealing with decision processes involving two decision makers with a hierarchical structure. They are characterised by the existence of two optimisation problems in which the constraint region of the upper level problem is implicitly determined by the lower level optimisation problem. In this paper we focus on the class of bilevel problems in which the upper level objective function is linear multiplicative, the lower level one is linear and the common constraint region is a bounded polyhedron. After replacing the lower level problem by its Karush–Kuhn–Tucker conditions, the existence of an extreme point which solves the problem is proved by using a penalty function approach. Besides, an algorithm based on the successive introduction of valid cutting planes is developed obtaining a global optimal solution. Finally, we generalise the problem by including upper level constraints which involve both level variables.     

Numerical algorithm for a class of constrained optimal control problems of switched systems

In this paper, we consider an optimal control problem of switched systems with continuous-time inequality constraints. Because of the complexity of such constraints and switching laws, it is difficult to solve this problem by standard optimization techniques. To overcome the difficulty, we adopt a bi-level algorithm to divide the problem into two nonlinear constrained optimization problems: one continuous and the other discrete. To solve the problem, we transform the inequality constraints into equality constraints which is smoothed using a twice continuously differentiable function and treated as a penalty function. On this basis, the smoothed problem can be solved by any second-order gradient algorithm, e.g., Newton’s Method. Finally, numerical examples show that our method is effective compared to existing algorithms.     

On solving simple bilevel programs with a nonconvex lower level program

In this paper, we consider a simple bilevel program where the lower level program is a nonconvex minimization problem with a convex set constraint and the upper level program has a convex set constraint. By using the value function of the lower level program, we reformulate the bilevel program as a single level optimization problem with a nonsmooth inequality constraint and a convex set constraint. To deal with such a nonsmooth and nonconvex optimization problem, we design a smoothing projected gradient algorithm for a general optimization problem with a nonsmooth inequality constraint and a convex set constraint. We show that, if the sequence of penalty parameters is bounded then any accumulation point is a stationary point of the nonsmooth optimization problem and, if the generated sequence is convergent and the extended Mangasarian-Fromovitz constraint qualification holds at the limit then the limit point is a stationary point of the nonsmooth optimization problem. We apply the smoothing projected gradient algorithm to the bilevel program if a calmness condition holds and to an approximate bilevel program otherwise. Preliminary numerical experiments show that the algorithm is efficient for solving the simple bilevel program.     

Proximal Point Methods for Lipschitz Functions on Hadamard Manifolds: Scalar and Vectorial Cases

In the second part of our study, we introduce the concept of global extended exactness of penalty and augmented Lagrangian functions, and derive the localization principle in the extended form. The main idea behind the extended exactness consists in an extension of the original constrained optimization problem by adding some extra variables, and then construction of a penalty/augmented Lagrangian function for the extended problem. This approach allows one to design extended penalty/augmented Lagrangian functions having some useful properties (such as smoothness), which their counterparts for the original problem might not possess. In turn, the global exactness of such extended merit functions can be easily proved with the use of the localization principle presented in this paper, which reduces the study of global exactness to a local analysis of a merit function based on sufficient optimality conditions and constraint qualifications. We utilize the localization principle in order to obtain simple necessary and sufficient conditions for the global exactness of the extended penalty function introduced by Huyer and Neumaier, and in order to construct a globally exact continuously differentiable augmented Lagrangian function for nonlinear semidefinite programming problems.     

一类逼近l1精确罚函数的罚函数

本文对可微非线性规划问题提出了一个渐近算法,它是基于一类逼近l1精确罚函数的罚函数而提出的,我们证明了算法所得的极小点列的聚点均为原问题的最优解,并在Mangasarian-Fromovitz约束条件下,证明了有限次迭代之后,所有迭代均为可行的,即迭代所得的极小点为可行点.     

Sequential penalty quadratic programming filter methods for nonlinear programming

Filter approaches, initially proposed by Fletcher and Leyffer in 2002, are recently attached importance to. If the objective function value or the constraint violation is reduced, this step is accepted by a filter, which is the basic idea of the filter. In this paper, the filter approach is employed in a sequential penalty quadratic programming (SlQP) algorithm which is similar to that of Yuan's. In every trial step, the step length is controlled by a trust region radius. In this work, our purpose is not to reduce the objective function and constraint violation. We reduce the degree of constraint violation and some function, and the function is closely related to the objective function. This algorithm requires neither Lagrangian multipliers nor the strong decrease condition. Meanwhile, in our SlQP filter there is no requirement of large penalty parameter. This method produces K-T points for the original problem.     

Outer Approximation Method Incorporating a Quadratic Approximation for a DC Programming Problem

In this paper, we propose an outer approximation method using two quadratic functions approximating the constraint functions of a DC programming problem. By analyzing the relation among the eigenvectors of the Hessian matrices of the constraint functions, the search direction for a feasible solution of the problem is determined. Moreover, to avoid line searches along similar directions, we incorporate a penalty function method in the algorithm.     

On the application of the residual method for the correction of inconsistent problems of convex programming

For the correction of a convex programming problem with potentially inconsistent constraint system (an improper problem), we apply the residual method, which is a standard regularization procedure for ill-posed optimization models. A problem statement typical for the residual method is reduced to a minimization problem for an appropriate penalty function. We apply two classical penalty functions: the quadratic penalty function and the exact Eremin-Zangwill penalty function. For each of the approaches, we establish convergence conditions and bounds for the approximation error.     

Existence of Exact Penalty and Its Stability for Nonconvex Constrained Optimization Problems in Banach Spaces

In this paper we use the penalty approach in order to study two constrained minimization problems. A penalty function is said to have the generalized exact penalty property if there is a penalty coefficient for which approximate solutions of the unconstrained penalized problem are close enough to approximate solutions of the corresponding constrained problem. In this paper we show that the generalized exact penalty property is stable under perturbations of cost functions, constraint functions and the right-hand side of constraints.     

On smoothing exact penalty functions for nonlinear constrained optimization problems

In the paper, we give a smoothing approximation to the nondifferentiable exact penalty function for nonlinear constrained optimization problems. Error estimations are obtained among the optimal objective function values of the smoothed penalty problems, of the nonsmooth penalty problem and of the original problem. An algorithm based on our smoothing function is given, which is showed to be globally convergent under some mild conditions.     

A Quasi-Newton Quadratic Penalty Method for Minimization Subject to Nonlinear Equality Constraints

We present a modified quadratic penalty function method for equality constrained optimization problems. The pivotal feature of our algorithm is that at every iterate we invoke a special change of variables to improve the ability of the algorithm to follow the constraint level sets. This change of variables gives rise to a suitable block diagonal approximation to the Hessian which is then used to construct a quasi-Newton method. We show that the complete algorithm is globally convergent. Preliminary computational results are reported.     

Optimal discrete-valued control computation

In this paper, we consider an optimal control problem in which the control takes values from a discrete set and the state and control are subject to continuous inequality constraints. By introducing auxiliary controls and applying a time-scaling transformation, we transform this optimal control problem into an equivalent problem subject to additional linear and quadratic constraints. The feasible region defined by these additional constraints is disconnected, and thus standard optimization methods struggle to handle these constraints. We introduce a novel exact penalty function to penalize constraint violations, and then append this penalty function to the objective. This leads to an approximate optimal control problem that can be solved using standard software packages such as MISER. Convergence results show that when the penalty parameter is sufficiently large, any local solution of the approximate problem is also a local solution of the original problem. We conclude the paper with some numerical results for two difficult train control problems.     

群零模正则化问题的等价Lipschitz优化模型

针对群零模正则化问题, 从零模函数的变分刻画入手, 将其等价地表示为带有 互补约束的数学规划问题(简称MPCC问题), 然后证明将互补约束直接罚到MPCC的目标函数而得到的罚问题是MPCC问题的全局精确罚. 此精确罚问题的目标函数不仅在可行集上全局Lipschitz连续而且还具有满意的双线性结构, 为设计群零模正则化问题的序列凸松弛算法提供了满意的等价Lipschitz优化模型.     

"准"精确惩罚函数法的渐近性分析

1引言众所周知,罚函数法在最优化理论与数值计算中占据着极其重要的位置,作为求解约束优化问题的一类重要方法,在上世纪五、六十年代曾经历一次发展高潮．近十几年来,伴随着对数障碍函数法在内点法中取得的成功,罚函数法的研究又呈现出一个小高潮[2,3,4]．在罚函数方法里,精确惩罚函数法有着非常吸引人的性质,即,当罚参数大于某个有限门槛值时,仅通过求解单个无约束罚问题便可得到原问题的最优解,从而省去了一般罚函数法解系列无约束优化问题的工作量．