Hyperbolic smoothing function method for minimax problems

In this article, an approach for solving finite minimax problems is proposed. This approach is based on the use of hyperbolic smoothing functions. In order to apply the hyperbolic smoothing we reformulate the objective function in the minimax problem and study the relationship between the original minimax and reformulated problems. We also study main properties of the hyperbolic smoothing function. Based on these results an algorithm for solving the finite minimax problem is proposed and this algorithm is implemented in general algebraic modelling system. We present preliminary results of numerical experiments with well-known nonsmooth optimization test problems. We also compare the proposed algorithm with the algorithm that uses the exponential smoothing function as well as with the algorithm based on nonlinear programming reformulation of the finite minimax problem.     

A derivative-free algorithm for systems of nonlinear inequalities

Recently a new derivative-free algorithm has been proposed for the solution of linearly constrained finite minimax problems. This derivative-free algorithm is based on a smoothing technique that allows one to take into account the non-smoothness of the max function. In this paper, we investigate, both from a theoretical and computational point of view, the behavior of the minmax algorithm when used to solve systems of nonlinear inequalities when derivatives are unavailable. In particular, we show an interesting property of the algorithm, namely, under some mild conditions regarding the regularity of the functions defining the system, it is possible to prove that the algorithm locates a solution of the problem after a finite number of iterations. Furthermore, under a weaker regularity condition, it is possible to show that an accumulation point of the sequence generated by the algorithm exists which is a solution of the system. Moreover, we carried out numerical experimentation and comparison of the method against a standard pattern search minimization method. The obtained results confirm that the good theoretical properties of the method correspond to interesting numerical performance. Moreover, the algorithm compares favorably with a standard derivative-free method, and this seems to indicate that extending the smoothing technique to pattern search algorithms can be beneficial.     

Algorithms with Adaptive Smoothing for Finite Minimax Problems

We present a new feedback precision-adjustment rule for use with a smoothing technique and standard unconstrained minimization algorithms in the solution of finite minimax problems. Initially, the feedback rule keeps a precision parameter low, but allows it to grow as the number of iterations of the resulting algorithm goes to infinity. Consequently, the ill-conditioning usually associated with large precision parameters is considerably reduced, resulting in more efficient solution of finite minimax problems.The resulting algorithms are very simple to implement, and therefore are particularly suitable for use in situations where one cannot justify the investment of time needed to retrieve a specialized minimax code, install it on one's platform, learn how to use it, and convert data from other formats. Our numerical tests show that the algorithms are robust and quite effective, and that their performance is comparable to or better than that of other algorithms available in the Matlab environment.     

Variable Programming: A Generalized Minimax Problem. Part II: Algorithms

In this part of the two-part series of papers, algorithms for solving some variable programming (VP) problems proposed in Part I are investigated. It is demonstrated that the non-differentiability and the discontinuity of the maximum objective function, as well as the summation objective function in the VP problems constitute difficulty in finding their solutions. Based on the principle of statistical mechanics, we derive smooth functions to approximate these non-smooth objective functions with specific activated feasible sets. By transforming the minimax problem and the corresponding variable programming problems into their smooth versions we can solve the resulting problems by some efficient algorithms for smooth functions. Relevant theoretical underpinnings about the smoothing techniques are established. The algorithms, in which the minimization of the smooth functions is carried out by the standard quasi-Newton method with BFGS formula, are tested on some standard minimax and variable programming problems. The numerical results show that the smoothing techniques yield accurate optimal solutions and that the algorithms proposed are feasible and efficient.This work was supported by the RGC grant CUHK 152/96H of the Hong Kong Research Grant Council.     

A nonmonotone Jacobian smoothing inexact Newton method for NCP

In this paper we propose Jacobian smoothing inexact Newton method for nonlinear complementarity problems (NCP) with derivative-free nonmonotone line search. This nonmonotone line search technique ensures globalization and is a combination of Grippo-Lampariello-Lucidi (GLL) and Li-Fukushima (LF) strategies, with the aim to take into account their advantages. The method is based on very well known Fischer-Burmeister reformulation of NCP and its smoothing Kanzow’s approximation. The mixed Newton equation, which combines the semismooth function with the Jacobian of its smooth operator, is solved approximately in every iteration, so the method belongs to the class of Jacobian smoothing inexact Newton methods. The inexact search direction is not in general a descent direction and this is the reason why nonmonotone scheme is used for globalization. Global convergence and local superlinear convergence of method are proved. Numerical performances are also analyzed and point out that high level of nonmonotonicity of this line search rule enables robust and efficient method.     

Error bounds of two smoothing approximations for semi-infinite minimax problems

In the paper we investigate smoothing method for solving semi-infinite minimax problems. Not like most of the literature in semi-infinite minimax problems which are concerned with the continuous time version（i.e., the one dimensional semi-infinite minimax problems）, the primary focus of this paper is on multi- dimensional semi-infinite minimax problems. The global error bounds of two smoothing approximations for the objective function are given and compared. It is proved that the smoothing approximation given in this paper can provide a better error bound than the existing one in literature.     

A dual algorithm for minimax problems

In this paper, a dual algorithm, based on a smoothing function of Bertsekas (1982), is established for solving unconstrained minimax problems. It is proven that a sequence of points, generated by solving a sequence of unconstrained minimizers of the smoothing function with changing parametert, converges with Q-superlinear rate to a Kuhn-Tucker point locally under some mild conditions. The relationship between the condition number of the Hessian matrix of the smoothing function and the parameter is studied, which also validates the convergence theory. Finally the numerical results are reported to show the effectiveness of this algorithm.     

A global optimization algorithm for generalized semi-infinite,continuous minimax with coupled constraints and bi-level problems

We propose an algorithm for the global optimization of three problem classes: generalized semi-infinite, continuous coupled minimax and bi-level problems. We make no convexity assumptions. For each problem class, we construct an oracle that decides whether a given objective value is achievable or not. If a given value is achievable, the oracle returns a point with a value better than or equal to the target. A binary search is then performed until the global optimum is obtained with the desired accuracy. This is achieved by solving a series of appropriate finite minimax and min-max-min problems to global optimality. We use Laplace’s smoothing technique and a simulated annealing approach for the solution of these problems. We present computational examples for all three problem classes.     

A truncated aggregate smoothing Newton method for minimax problems

Aggregate function is a useful smoothing function to the max-function of some smooth functions and has been used to solve minimax problems, linear and nonlinear programming, generalized complementarity problems, etc. The aggregate function is a single smooth but complex function, its gradient and Hessian calculations are time-consuming. In this paper, a truncated aggregate smoothing stabilized Newton method for solving minimax problems is presented. At each iteration, only a small subset of the components in the max-function are aggregated, hence the number of gradient and Hessian calculations is reduced dramatically. The subset is adaptively updated with some truncating criterions, concerning only with computation of function values and not their gradients or Hessians, to guarantee the global convergence and, for the inner iteration, locally quadratic convergence with as few computational cost as possible. Numerical results show the efficiency of the proposed algorithm.     

Nonmonotone smoothing Broyden-like method for generalized nonlinear complementarity problems

Based on a new symmetrically perturbed smoothing function, the generalized nonlinear complementarity problem defined on a polyhedral cone is reformulated as a system of smoothing equations. Then we suggest a new nonmonotone derivative-free line search and combine it into the smoothing Broyden-like method. The proposed algorithm contains the usual monotone line search as a special case and can overcome the difficult of smoothing Newton methods in solving the smooth equations to some extent. Under mild conditions, we prove that the proposed algorithm has global and local superlinear convergence. Furthermore, the algorithm is locally quadratically convergent under suitable assumptions. Preliminary numerical results are also reported.     

基于二次多项式的积极集光滑化max函数及其在无约束minimax问题中的应用

本文基于分段二次多项式方程,构造了一种积极集策略的光滑化max函数.通过给出与光滑化max函数相关的分量函数指标集的直接计算方法,将分段二次多项式方程转化为一般二次多项式方程.利用二次多项式方程根的性质,给出了该光滑化max函数的稳定计算策略,证明了其具有一阶光滑性,其梯度函数具有局部Lipschitz连续性和强半光滑性.该光滑化max函数仅与函数值较大的分量函数相关,适用于含分量函数较多且复杂的max函数的问题.为了验证其效率,本文基于该函数构造了一种解含多个复杂分量函数的无约束minimax问题的光滑化算法,数值实验表明了该光滑化max函数的可行性及有效性.     

A smoothing and regularization Broyden-like method for nonlinear inequalities

In this paper, we consider the smoothing and regularization Broyden-like algorithm for the system of nonlinear inequalities. By constructing a new smoothing function $\phi(\mu,a)=\frac{1}{2}(a+\mu(\ln2+\ln(1+\cosh\frac{a}{\mu})))$ , the problem is approximated via a family of parameterized smooth equations H(μ,ε,x)=0. A smoothing and regularization Broyden-like algorithm with a non-monotone linear search is proposed for solving the system of nonlinear inequalities based on the new smoothing function. The global convergence of the algorithm is established under suitable assumptions. In addition, the smoothing parameter μ and the regularization parameter ε in our algorithm are viewed as two different independent variables. Preliminary numerical results show the efficiency of the algorithm and reveal that the regularization parameter ε in our algorithm plays an important role in numerical improvement, hence, our algorithm seems to be simpler and more easily implemented compared to many previous methods.     

A non-interior continuation method for generalized linear complementarity problems

In this paper, we propose a non-interior continuation method for solving generalized linear complementarity problems (GLCP) introduced by Cottle and Dantzig. The method is based on a smoothing function derived from the exponential penalty function first introduced by Kort and Bertsekas for constrained minimization. This smoothing function can also be viewed as a natural extension of Chen-Mangasarian’s neural network smooth function. By using the smoothing function, we approximate GLCP as a family of parameterized smooth equations. An algorithm is presented to follow the smoothing path. Under suitable assumptions, it is shown that the algorithm is globally convergent and local Q-quadratically convergent. Few preliminary numerical results are also reported. Received September 3, 1997 / Revised version received April 27, 1999?Published online July 19, 1999     

A new family of hybrid three-term conjugate gradient methods with applications in image restoration

In this paper, based on the hybrid conjugate gradient method and the convex combination technique, a new family of hybrid three-term conjugate gradient methods are proposed for solving unconstrained optimization. The conjugate parameter in the search direction is a hybrid of Dai-Yuan conjugate parameter and any one. The search direction then is the sum of the negative gradient direction and a convex combination in relation to the last search direction and the gradient at the previous iteration. Without choosing any specific conjugate parameters, we show that the search direction generated by the family always possesses the descent property independent of line search technique, and that it is globally convergent under usual assumptions and the weak Wolfe line search. To verify the effectiveness of the presented family, we further design a specific conjugate parameter, and perform medium-large-scale numerical experiments for smooth unconstrained optimization and image restoration problems. The numerical results show the encouraging efficiency and applicability of the proposed methods even compared with the state-of-the-art methods.

     

Smoothing Method for Minimax Problems

In this paper, we propose a smoothing method for minimax problem. The method is based on the exponential penalty function of Kort and Bertsekas for constrained optimization. Under suitable condition, the method is globally convergent. Preliminary numerical experiments indicate the promising of the algorithm.     

A new hybrid method for nonlinear complementarity problems

In this paper, a new hybrid method is proposed for solving nonlinear complementarity problems (NCP) with P ₀ function. In the new method, we combine a smoothing nonmonotone trust region method based on a conic model and line search techniques. We reformulate the NCP as a system of semismooth equations using the Fischer-Burmeister function. Using Kanzow’s smooth approximation function to construct the smooth operator, we propose a smoothing nonmonotone trust region algorithm of a conic model for solving the NCP with P ₀ functions. This is different from the classical trust region methods, in that when a trial step is not accepted, the method does not resolve the trust region subproblem but generates an iterative point whose steplength is defined by a line search. We prove that every accumulation point of the sequence generated by the algorithm is a solution of the NCP. Under a nonsingularity condition, the superlinear convergence of the algorithm is established without a strict complementarity condition.     

垂直线性互补问题的一种光滑算法

文中给出了垂直线性互补问题的一个新的光滑价值函数,不同于光滑化方法中的价值函数,它不包含任何必须趋向零的参数,因此算法中不涉及参数调整步骤,而且具有良好的强制性.基此价值函数,提出了求解垂直线性互补问题的一种阻尼Newton类算法,并证明了该算法对竖块P0+R0矩阵的垂直线性互补问题具有全局收敛性;当解满足相当于BD-正则条件时,算法具有局部二次收敛性;在不增加额外校正步骤(算法的每个迭代步只求解一个Newton方程)的情形下,算法对竖块P-矩阵垂直线性互补问题(无须假设严格互补),具有有限步收敛性.数值实验结果令人满意.     

核实数据下非线性半参数EV模型的估计

本文研究了核实数据下的协变量带有测量误差的非线性半参数EV模型.在不假定测量误差结构的情形下,利用最小二乘方法和核光滑技术,构造了非线性函数中未知参数的两种估计,证明了未知参数估计的渐近正态性.通过数值模拟说明所提估计方法在有限样本下的有效性.     

数据挖掘中聚类中心问题的光滑化和填充函数方法

本文提出了数据挖掘中求解聚类中心问题的一种新方法.这类问题属于非凸非光滑全局最优化问题.我们首先利用光滑化方法将非光滑聚类函数用光滑函数逼近,然后对光滑化问题利用填充函数搜索其全局最优点.对不同数据库的数值试验表明,本文提出的算法是可行和有效的.     

One-step smoothing Newton method for solving the mixed complementarity problem with a function

The mixed complementarity problem (denote by MCP(F)) can be reformulated as the solution of a smooth system of equations. In the paper, based on a perturbed mid function, we propose a new smoothing function, which has an important property, not satisfied by many other smoothing function. The existence and continuity of a smooth path for solving the mixed complementarity problem with a P₀ function are discussed. Then we presented a one-step smoothing Newton algorithm to solve the MCP with a P₀ function. The global convergence of the proposed algorithm is verified under mild conditions. And by using the smooth and semismooth technique, the rate of convergence of the method is proved under some suitable assumptions.