A smoothing augmented Lagrangian method for solving simple bilevel programs

In this paper, we design a numerical algorithm for solving a simple bilevel program where the lower level program is a nonconvex minimization problem with a convex set constraint. We propose to solve a combined problem where the first order condition and the value function are both present in the constraints. Since the value function is in general nonsmooth, the combined problem is in general a nonsmooth and nonconvex optimization problem. We propose a smoothing augmented Lagrangian method for solving a general class of nonsmooth and nonconvex constrained optimization problems. We show that, if the sequence of penalty parameters is bounded, then any accumulation point is a Karush-Kuch-Tucker (KKT) point of the nonsmooth optimization problem. The smoothing augmented Lagrangian method is used to solve the combined problem. Numerical experiments show that the algorithm is efficient for solving the simple bilevel program.     

On solving simple bilevel programs with a nonconvex lower level program

In this paper, we consider a simple bilevel program where the lower level program is a nonconvex minimization problem with a convex set constraint and the upper level program has a convex set constraint. By using the value function of the lower level program, we reformulate the bilevel program as a single level optimization problem with a nonsmooth inequality constraint and a convex set constraint. To deal with such a nonsmooth and nonconvex optimization problem, we design a smoothing projected gradient algorithm for a general optimization problem with a nonsmooth inequality constraint and a convex set constraint. We show that, if the sequence of penalty parameters is bounded then any accumulation point is a stationary point of the nonsmooth optimization problem and, if the generated sequence is convergent and the extended Mangasarian-Fromovitz constraint qualification holds at the limit then the limit point is a stationary point of the nonsmooth optimization problem. We apply the smoothing projected gradient algorithm to the bilevel program if a calmness condition holds and to an approximate bilevel program otherwise. Preliminary numerical experiments show that the algorithm is efficient for solving the simple bilevel program.     

A smoothing trust region filter algorithm for nonsmooth least squares problems

We propose a smoothing trust region filter algorithm for nonsmooth nonconvex least squares problems. We present convergence theorems of the proposed algorithm to a Clarke stationary point or a global minimizer of the objective function under certain conditions. Preliminary numerical experiments show the efficiency of the proposed algorithm for finding zeros of a system of polynomial equations with high degrees on the sphere and solving differential variational inequalities.     

Nonsmooth nonconvex optimization approach to clusterwise linear regression problems

Clusterwise regression consists of finding a number of regression functions each approximating a subset of the data. In this paper, a new approach for solving the clusterwise linear regression problems is proposed based on a nonsmooth nonconvex formulation. We present an algorithm for minimizing this nonsmooth nonconvex function. This algorithm incrementally divides the whole data set into groups which can be easily approximated by one linear regression function. A special procedure is introduced to generate a good starting point for solving global optimization problems at each iteration of the incremental algorithm. Such an approach allows one to find global or near global solution to the problem when the data sets are sufficiently dense. The algorithm is compared with the multistart Späth algorithm on several publicly available data sets for regression analysis.     

数据挖掘中聚类中心问题的光滑化和填充函数方法

本文提出了数据挖掘中求解聚类中心问题的一种新方法.这类问题属于非凸非光滑全局最优化问题.我们首先利用光滑化方法将非光滑聚类函数用光滑函数逼近,然后对光滑化问题利用填充函数搜索其全局最优点.对不同数据库的数值试验表明,本文提出的算法是可行和有效的.     

Two-Phase Image Segmentation by Nonconvex Nonsmooth Models with Convergent Alternating Minimization Algorithms

Two-phase image segmentation is a fundamental task to partition an image into foreground and background. In this paper, two types of nonconvex and nonsmooth regularization models are proposed for basic two-phase segmentation. They extend the convex regularization on the characteristic function on the image domain to the nonconvex case, which are able to better obtain piecewise constant regions with neat boundaries. By analyzing the proposed non-Lipschitz model, we combine the proximal alternating minimization framework with support shrinkage and linearization strategies to design our algorithm. This leads to two alternating strongly convex subproblems which can be easily solved. Similarly, we present an algorithm without support shrinkage operation for the nonconvex Lipschitz case. Using the Kurdyka-Łojasiewicz property of the objective function, we prove that the limit point of the generated sequence is a critical point of the original nonconvex nonsmooth problem. Numerical experiments and comparisons illustrate the effectiveness of our method in two-phase image segmentation.     

基于凝聚函数的互补问题的光滑化算法

对于不可微的"极大值"形式的函数,可以利用凝聚函数对其进行光滑逼近.借助这个技术,给出了求解线性互补问题的光滑方程组算法.首先是将互补问题转化为等价的非光滑方程组,再利用凝聚函数进行光滑逼近,从而转化为光滑方程组的求解问题.通过一些考题对这个算法进行了数值试验,结果显示了该算法的有效性和稳定性.     

A feasible SQP-GS algorithm for nonconvex, nonsmooth constrained optimization

The gradient sampling (GS) algorithm for minimizing a nonconvex, nonsmooth function was proposed by Burke et al. (SIAM J Optim 15:751–779, 2005), whose most interesting feature is the use of randomly sampled gradients instead of subgradients. In this paper, combining the GS technique with the sequential quadratic programming (SQP) method, we present a feasible SQP-GS algorithm that extends the GS algorithm to nonconvex, nonsmooth constrained optimization. The proposed algorithm generates a sequence of feasible iterates, and guarantees that the objective function is monotonically decreasing. Global convergence is proved in the sense that, with probability one, every cluster point of the iterative sequence is stationary for the improvement function. Finally, some preliminary numerical results show that the proposed algorithm is effective.     

Cutting Sphere Algorithm

We present a generalization of the conventional cutting plane algorithm for the solution of nonconvex optimization problems with nonsmooth inequality constraints. The cuts are effected using spheres rather than hyperplanes.     

Nonconvex energy functions,related eigenvalue hemivariational inequalities on the sphere and applications

The present paper deals with a new type of eigenvalue problems arising in problems involving nonconvex nonsmooth energy functions. They lead to the search of critical points (e.g. local minima) for nonconvex nonsmooth potential functions which in turn give rise to hemivariational inequalities. For this type of variational expressions the eigenvalue problem is studied here concerning the existence and multiplicity of solutions by applying a critical point theory appropriate for nonsmooth nonconvex functionals.     

Nonsmooth Bundle Trust-region Algorithm with Applications to Robust Stability

We propose a bundle trust-region algorithm to minimize locally Lipschitz functions which are potentially nonsmooth and nonconvex. We prove global convergence of our method and show by way of an example that the classical convergence argument in trust-region methods based on the Cauchy point fails in the nonsmooth setting. Our method is tested experimentally on three problems in automatic control.     

求解摩擦接触问题的一个非内点光滑化算法

给出了一个求解三维弹性有摩擦接触问题的新算法,即基于NCP函数的非内点光滑化算法．首先通过参变量变分原理和参数二次规划法,将三维弹性有摩擦接触问题的分析归结为线性互补问题的求解;然后利用NCP函数,将互补问题的求解转换为非光滑方程组的求解;再用凝聚函数对其进行光滑化,最后用NEWTON法解所得到的光滑非线性方程组．方法具有易于理解及实现方便等特点．通过线性互补问题的数值算例及接触问题实例证实了该算法的可靠性与有效性．     

Epi-convergence Properties of Smoothing by Infimal Convolution

This paper concerns smoothing by infimal convolution for two large classes of functions: convex, proper and lower semicontinous as well as for (the nonconvex class of) convex-composite functions. The smooth approximations are constructed so that they epi-converge (to the underlying nonsmooth function) and fulfill a desirable property with respect to graph convergence of the gradient mappings to the subdifferential of the original function under reasonable assumptions. The close connection between epi-convergence of the smoothing functions and coercivity properties of the smoothing kernel is established.     

二阶锥规划一个超线性收敛的非内部连续化算法

基于非光滑向量值最小函数的一个新光滑函数, 建立了二阶锥规划一个超线性收敛的非内部连续化算法. 该算法的特点如下: 首先, 初始点任意; 其次, 每次迭代只需求解一个线性方程组即可得到搜索方向; 最后, 在无严格互补假设下, 获得算法的全局收敛性、强收敛性和超线性收敛性. 数值结果表明算法是有效的.     

A proximal algorithm with quasi distance. Application to habit's formation

We consider a proximal algorithm with quasi distance applied to nonconvex and nonsmooth functions involving analytic properties for a minimization problem. We show the behavioural importance of this proximal point model for the formation of habit in Decision and Making Sciences. The convergence of the sequence generated by our algorithm to a critical-limit point is guaranteed under standard conditions of coercivity and by using the Kurdyka–?ojasiewicz inequality. We present a definition of habit that contemplates that kind of convergence.     

A new nonsmooth optimization algorithm for minimum sum-of-squares clustering problems

The minimum sum-of-squares clustering problem is formulated as a problem of nonsmooth, nonconvex optimization, and an algorithm for solving the former problem based on nonsmooth optimization techniques is developed. The issue of applying this algorithm to large data sets is discussed. Results of numerical experiments have been presented which demonstrate the effectiveness of the proposed algorithm.     

Regularization in the Mosolov and Myasnikov problem with boundary friction

We propose an iterative algorithm for solving a semicoercive nonsmooth variational inequality. The algorithm is based on the stepwise partial smoothing of the minimized functional and an iterative proximal regularization method.We obtain a solution to the variational Mosolov and Myasnikov problem with boundary friction as a limit point of a sequence of solutions to stable auxiliary problems.     

Global Convergence of a Robust Smoothing SQP Method for Semi-Infinite Programming

The semi-infinite programming (SIP) problem is a program with infinitely many constraints. It can be reformulated as a nonsmooth nonlinear programming problem with finite constraints by using an integral function. Due to the nondifferentiability of the integral function, gradient-based algorithms cannot be used to solve this nonsmooth nonlinear programming problem. To overcome this difficulty, we present a robust smoothing sequential quadratic programming (SQP) algorithm for solving the nonsmooth nonlinear programming problem. At each iteration of the algorthm, we need to solve only a quadratic program that is always feasible and solvable. The global convergence of the algorithm is established under mild conditions. Numerical results are given. Communicated by F. Giannessi His work was supported by the Hong Kong Research Grant Council His work was supported by the Australian Research Council.     

Global convergence of proximal iteratively reweighted algorithm

In this paper, we investigate the convergence of the proximal iteratively reweighted algorithm for a class of nonconvex and nonsmooth problems. Such problems actually include numerous models in the area of signal processing and machine learning research. Two extensions of the algorithm are also studied. We provide a unified scheme for these three algorithms. With the Kurdyka–?ojasiewicz property, we prove that the unified algorithm globally converges to a critical point of the objective function.     

非光滑非凸多目标优化不动点方法

本文首次讨论了用不动点刻划的不可微多目标优化的最优性必要条件和充分条件，并研究了不动点算法求解此问题的方法及大范围收敛性．为不可微多目标优化研究提供了另一条新的途径．