Enhanced speciation in particle swarm optimization for multi-modal problems

In this paper, we present a novel multi-modal optimization algorithm for finding multiple local optima in objective function surfaces. We build from Species-based particle swarm optimization (SPSO) by using deterministic sampling to generate new particles during the optimization process, by implementing proximity-based speciation coupled with speciation of isolated particles, and by including “turbulence regions” around already found solutions to prevent unnecessary function evaluations. Instead of using error threshold values, the new algorithm uses the particle’s experience, geometric mean, and “exclusion factor” to detect local optima and stop the algorithm. The performance of each extension is assessed with leave-it-out tests, and the results are discussed. We use the new algorithm called Isolated-Speciation-based particle swarm optimization (ISPSO) and a benchmark algorithm called Niche particle swarm optimization (NichePSO) to solve a six-dimensional rainfall characterization problem for 192 rain gages across the United States. We show why it is important to find multiple local optima for solving this real-world complex problem by discussing its high multi-modality. Solutions found by both algorithms are compared, and we conclude that ISPSO is more reliable than NichePSO at finding optima with a significantly lower objective function value.     

A new exclusion test for finding the global minimum

Exclusion algorithms have been used recently to find all solutions of a system of nonlinear equations or to find the global minimum of a function over a compact domain. These algorithms are based on a minimization condition that can be applied to each cell in the domain. In this paper, we consider Lipschitz functions of order α and give a new minimization condition for the exclusion algorithm. Furthermore, convergence and complexity results are presented for such algorithm.     

Nonsmooth exclusion test for finding all solutions of nonlinear equations

A new approach is proposed for finding all real solutions of systems of nonlinear equations with bound constraints. The zero finding problem is converted to a global optimization problem whose global minima with zero objective value, if any, correspond to all solutions of the original problem. A branch-and-bound algorithm is used with McCormick’s nonsmooth convex relaxations to generate lower bounds. An inclusion relation between the solution set of the relaxed problem and that of the original nonconvex problem is established which motivates a method to generate automatically, starting points for a local Newton-type method. A damped-Newton method with natural level functions employing the restrictive monotonicity test is employed to find solutions robustly and rapidly. Due to the special structure of the objective function, the solution of the convex lower bounding problem yields a nonsmooth root exclusion test which is found to perform better than earlier interval-analysis based exclusion tests. Both the componentwise Krawczyk operator and interval-Newton operator with Gauss-Seidel based root inclusion and exclusion tests are also embedded in the proposed algorithm to refine the variable bounds for efficient fathoming of the search space. The performance of the algorithm on a variety of test problems from the literature is presented, and for most of them, the first solution is found at the first iteration of the algorithm due to the good starting point generation.     

A new penalty-free-type algorithm based on trust region techniques

In this paper, we propose a new penalty-free-type method for nonlinear equality constrained problems. The new algorithm uses trust region framework and feasibility safeguarding technique. Moreover, it has no choice of penalty parameter and penalty function as a merit function, and it does not use the filter technique to avoid the penalty function either. We analyze the global convergence of the main algorithm under the standard assumptions. The preliminary numerical tests are reported.     

A semismooth equation approach to the solution of nonlinear complementarity problems

In this paper we present a new algorithm for the solution of nonlinear complementarity problems. The algorithm is based on a semismooth equation reformulation of the complementarity problem. We exploit the recent extension of Newton's method to semismooth systems of equations and the fact that the natural merit function associated to the equation reformulation is continuously differentiable to develop an algorithm whose global and quadratic convergence properties can be established under very mild assumptions. Other interesting features of the new algorithm are an extreme simplicity along with a low computational burden per iteration. We include numerical tests which show the viability of the approach.     

Adaptive meshless refinement schemes for RBF-PUM collocation

In this paper we present an adaptive discretization technique for solving elliptic partial differential equations via a collocation radial basis function partition of unity method. In particular, we propose a new adaptive scheme based on the construction of an error indicator and a refinement algorithm, which used together turn out to be ad-hoc strategies within this framework. The performance of the adaptive meshless refinement scheme is assessed by numerical tests.     

一个解大规模无约束优化问题的全局梯度法(英文)

本文提出一种新的解大规模无约束优化问题的全局收敛的梯度法.新算法沿着负梯度方向选择步长,而初始步长根据目标函数的海赛矩阵的近似数量矩阵来确定.理论上证明了新算法产生的点列的每个聚点都是稳定的,数值试验表明新算法是可靠且有效的.     

Approximating the zeros of analytic functions by the exclusion algorithm

We give a practical version of the exclusion algorithm for localizing the zeros of an analytic function and in particular of a polynomial in a compact of . We extend the real exclusion algorithm to a Jordan curve and give a method which excludes discs without any zero. The result of this algorithm is a set of discs arbitrarily small which contains the zeros of the analytic function.     

Categorical data clustering with automatic selection of cluster number

In this paper, we investigate the problem of determining the number of clusters in the k-modes based categorical data clustering process. We propose a new categorical data clustering algorithm with automatic selection of k. The new algorithm extends the k-modes clustering algorithm by introducing a penalty term to the objective function to make more clusters compete for objects. In the new objective function, we employ a regularization parameter to control the number of clusters in a clustering process. Instead of finding k directly, we choose a suitable value of regularization parameter such that the corresponding clustering result is the most stable one among all the generated clustering results. Experimental results on synthetic data sets and the real data sets are used to demonstrate the effectiveness of the proposed algorithm.     

A deterministic approach to global box-constrained optimization

A deterministic global optimization algorithm for box-constrained problems is presented. The proposed approach is based on well-known non-uniform space covering technique. In the paper this approach is further elaborated. We propose a new techniques that enables a significant reduction of the search space by means of dropping parts of processed boxes. Also a new quadratic underestimation for the objective function based on hessian eigenvalues bounds is presented. It is shown how this underestimation can be improved by exploiting the first-order optimality conditions. In the experimental section we compare the proposed approach with existing methods and programming tools. Numerical tests indicate that the proposed algorithm is highly competitive with considered methods.     

A nonmonotone derivative-free algorithm for nonlinear complementarity problems based on the new generalized penalized Fischer–Burmeister merit function

In this paper, we propose a new generalized penalized Fischer–Burmeister merit function, and show that the function possesses a system of favorite properties. Moreover, for the merit function, we establish the boundedness of level set under a weaker condition. We also propose a derivative-free algorithm for nonlinear complementarity problems with a nonmonotone line search. More specifically, we show that the proposed algorithm is globally convergent and has a locally linear convergence rate. Numerical comparisons are also made with the merit function used by Chen (J Comput Appl Math 232:455–471, 2009), which confirm the superior behaviour of the new merit function.     

Path-following gradient-based decomposition algorithms for separable convex optimization

A new decomposition optimization algorithm, called path-following gradient-based decomposition, is proposed to solve separable convex optimization problems. Unlike path-following Newton methods considered in the literature, this algorithm does not require any smoothness assumption on the objective function. This allows us to handle more general classes of problems arising in many real applications than in the path-following Newton methods. The new algorithm is a combination of three techniques, namely smoothing, Lagrangian decomposition and path-following gradient framework. The algorithm decomposes the original problem into smaller subproblems by using dual decomposition and smoothing via self-concordant barriers, updates the dual variables using a path-following gradient method and allows one to solve the subproblems in parallel. Moreover, compared to augmented Lagrangian approaches, our algorithmic parameters are updated automatically without any tuning strategy. We prove the global convergence of the new algorithm and analyze its convergence rate. Then, we modify the proposed algorithm by applying Nesterov’s accelerating scheme to get a new variant which has a better convergence rate than the first algorithm. Finally, we present preliminary numerical tests that confirm the theoretical development.     

无线搜索下仿射尺度算法的收敛性

本文我们考虑求解边界约束优化问题的一个仿射尺度算法。该方法的主要特点是在每次迭代过程中不需要任何线搜索,从而避免了多次调用目标函数的计算。在一定条件下,获得了算法的全局收敛性,数值测试证明了方法的有效性。     

Nonlinear programming without a penalty function

In this paper the solution of nonlinear programming problems by a Sequential Quadratic Programming (SQP) trust-region algorithm is considered. The aim of the present work is to promote global convergence without the need to use a penalty function. Instead, a new concept of a “filter” is introduced which allows a step to be accepted if it reduces either the objective function or the constraint violation function. Numerical tests on a wide range of test problems are very encouraging and the new algorithm compares favourably with LANCELOT and an implementation of Sl₁QP. Received: October 17, 1997 / Accepted: August 17, 2000?Published online September 3, 2001     

一类连续函数模拟退火算法及其收敛性分析

高维连续函数的全局优化问题普遍存在于计算生物学、计算化学等领域．针对这类问题和现有连续函数模拟退火算法的某些不足,本文给出了一类改进的模拟退火算法．采用一种简单的方法证明了算法的全局收敛性．数值结果表明,对于高维连续函数,该算法能够快速有效地收敛到全局最优点,比较了两种新解产生方法的试验结果。     

On the eigenvalues of the saddle point matrices discretized from Navier–Stokes equations

In this paper, we proposed an implementation of stochastic perturbation of reduced gradient and bisection (SPRGB) method for optimizing a non-convex differentiable function subject to linear equality constraints and non-negativity bounds on the variables. In particular, at each iteration, we compute a search direction by reduced gradient, and optimal line search by bisection algorithm along this direction yields a decrease in the objective value. SPRGB method is desired to establish the global convergence of the algorithm. An implementation and tests of SPRGB algorithm are given, and some numerical results of large-scale problems are presented, which show the efficient of this approach.     

Two tests for heteroscedasticity in nonparametric regression

In this paper, two new tests for heteroscedasticity in nonparametric regression are presented and compared. The first of these tests consists in first estimating nonparametrically the unknown conditional variance function and then using a classical least-squares test for a general linear model to test whether this function is a constant. The second test is based on using an overall distance between a nonparametric estimator of the conditional variance function and a parametric estimator of the variance of the model under the assumption of homoscedasticity. A bootstrap algorithm is used to approximate the distribution of this test statistic. Extended versions of both procedures in two directions, first, in the context of dependent data, and second, in the case of testing if the variance function is a polynomial of a certain degree, are also described. A broad simulation study is carried out to illustrate the finite sample performance of both tests when the observations are independent and when they are dependent.     

A Conic Trust-Region Method for Nonlinearly Constrained Optimization

Trust-region methods are powerful optimization methods. The conic model method is a new type of method with more information available at each iteration than standard quadratic-based methods. Can we combine their advantages to form a more powerful method for constrained optimization? In this paper we give a positive answer and present a conic trust-region algorithm for non-linearly constrained optimization problems. The trust-region subproblem of our method is to minimize a conic function subject to the linearized constraints and the trust region bound. The use of conic functions allows the model to interpolate function values and gradient values of the Lagrange function at both the current point and previous iterate point. Since conic functions are the extension of quadratic functions, they approximate general nonlinear functions better than quadratic functions. At the same time, the new algorithm possesses robust global properties. In this paper we establish the global convergence of the new algorithm under standard conditions.     

Bound constrained interval global optimization in the COCONUT Environment

We introduce a new interval global optimization method for solving bound constrained problems. The method originates from a small standalone software and is implemented in the COCONUT Environment, a framework designed for the development of complex algorithms, containing numerous state-of-the-art methods in a common software platform. The original algorithm is enhanced by various new methods implemented in COCONUT, regarding both interval function evaluations (such as first and second order derivatives with backward automatic differentiation, slopes, slopes of derivatives, bicentered forms, evaluations on the Karush–John conditions, etc.) and algorithmic elements (inclusion/exclusion boxes, local search, constraint propagation). This resulted in a substantial performance increase as compared to the original code. During the selection of the best combination of options, we performed comparison tests that gave empirical answers to long-lasting algorithmic questions (such as whether to use interval gradients or use slopes instead), that have never been studied numerically in such detail before. The new algorithm, called coco_gop_ex, was tested against the prestigious BARON software on an extensive set of bound constrained problems. We found that in addition to accepting a wider class of bound constrained problems and providing more output information (by locating all global minimizers), coco_gop_ex is competitive with BARON in terms of the solution success rates (with the exception of a set of nonlinear least squares problems), and it often outperforms BARON in running time. In particular, coco_gop_ex was around 21 % faster on average over the set of problems solved by both software systems.