Parallel Characteristical Algorithms for Solving Problems of Global Optimization

A class of parallel characteristical algorithms for global optimization ofone-dimensional multiextremal functions is introduced. General convergence andefficiency conditions for the algorithms of the class introduced areestablished. A generalization for the multidimensional case is considered.Examples of parallel characteristical algorithms and numerical experiments arepresented.     

New Inexact Line Search Method for Unconstrained Optimization

We propose a new inexact line search rule and analyze the global convergence and convergence rate of related descent methods. The new line search rule is similar to the Armijo line-search rule and contains it as a special case. We can choose a larger stepsize in each line-search procedure and maintain the global convergence of related line-search methods. This idea can make us design new line-search methods in some wider sense. In some special cases, the new descent method can reduce to the Barzilai and Borewein method. Numerical results show that the new line-search methods are efficient for solving unconstrained optimization problems. The work was supported by NSF of China Grant 10171054, Postdoctoral Fund of China, and K. C. Wong Postdoctoral Fund of CAS Grant 6765700. The authors thank the anonymous referees for constructive comments and suggestions that greatly improved the paper.     

一种修正的求约束总极值的积分-水平集方法

对于有约束的全局最优化问题,在Chew-Zheng的《Integral Global Optimization》和邬冬华等的《一种修正的求总极值的积分-水平集方法的实现算法收敛性》的基础上,给出一种修正的求约束总极值的积分-水平集方法,它同样具有修正的求总极值的积分-水平集方法的两个特点: 1) 每一步构造一个新函数,它与原目标函数具有相同的总极值; 2) 避免了郑权算法在一般情况下,由于水平集不易求得而造成难以求出水平集的困难．同时给出了其实现算法,并证明了算法的收敛性．     

求可测函数的总体极小值方法的收敛性分析

[1]中提出了求解连续函数f(x)总体极小值的均值算法，并证明了算法的全局收敛性．若假设f(t)是定义在某可测集G上的可测函数，本证明了均值算法产生的迭代序列全局收敛到f(t)的本质极小值，若进一步假设函数f(t)满足测度Lipschitz条件，还证明了求可测函数的均值算法是线性收敛的．     

New Inexact Parallel Variable Distribution Algorithms

We consider the recently proposed parallel variable distribution(PVD) algorithm of Ferris and Mangasarian [4] for solvingoptimization problems in which the variables are distributed amongp processors. Each processor has the primary responsibility forupdating its block of variables while allowing the remainingsecondary variables tochange in a restricted fashion along some easily computable directions.We propose useful generalizationsthat consist, for the general unconstrained case, of replacing exact global solution ofthe subproblems by a certain natural sufficient descent condition, and,for the convex case, of inexact subproblem solution in thePVD algorithm. These modifications are the key features ofthe algorithm that has not been analyzed before.The proposed modified algorithms are more practical andmake it easier to achieve good load balancing among the parallelprocessors.We present a general framework for the analysis of thisclass of algorithms and derive some new and improved linear convergence resultsfor problems with weak sharp minima of order 2 and strongly convex problems.We also show that nonmonotone synchronization schemesare admissible, which further improves flexibility of PVD approach.     

无线搜索共轭梯度法的全局收敛性

In this paper, a new steplength formula is proposed for unconstrained optimization,which can determine the step-size only by one step and avoids the line search step. Global convergence of the five well-known conjugate gradient methods with this formula is analyzed,and the corresponding results are as follows:(1) The DY method globally converges for a strongly convex LC~1 objective function;(2) The CD method, the FR method, the PRP method and the LS method globally converge for a general, not necessarily convex, LC~1 objective function.     

Simulated Annealing Algorithms for Continuous Global Optimization: Convergence Conditions

In this paper, simulated annealing algorithms for continuous global optimization are considered. After a review of recent convergence results from the literature, a class of algorithms is presented for which strong convergence results can be proved without introducing assumptions which are too restrictive. The main idea of the paper is that of relating both the temperature value and the support dimension of the next candidate point, so that they are small at points with function value close to the current record and bounded away from zero otherwise.     

Parallel Synchronous and Asynchronous Space-Decomposition Algorithms for Large-Scale Minimization Problems

Three parallel space-decomposition minimization (PSDM) algorithms, based on the parallel variable transformation (PVT) and the parallel gradient distribution (PGD) algorithms (O.L. Mangasarian, SIMA Journal on Control and Optimization, vol. 33, no. 6, pp. 1916–1925.), are presented for solving convex or nonconvex unconstrained minimization problems. The PSDM algorithms decompose the variable space into subspaces and distribute these decomposed subproblems among parallel processors. It is shown that if all decomposed subproblems are uncoupled of each other, they can be solved independently. Otherwise, the parallel algorithms presented in this paper can be used. Numerical experiments show that these parallel algorithms can save processor time, particularly for medium and large-scale problems. Up to six parallel processors are connected by Ethernet networks to solve four large-scale minimization problems. The results are compared with those obtained by using sequential algorithms run on a single processor. An application of the PSDM algorithms to the training of multilayer Adaptive Linear Neurons (Madaline) and a new parallel architecture for such parallel training are also presented.     

积分——水平集求全局最优的遗传算法实现

Chew Soo Hong,Zheng Q uan提出了一个积分——水平集求全局最优的概念性算法及M on te-C ar-lo随机投点的实现途径,并在很多实际问题中得到了很好的应用,但这一实现算法的收敛性是个未解决的问题.利用近年来广泛应用的遗传算法,给出了这一算法的另一种实现途径,并从理论和数值两个方面验证了算法的可行性.     

Convergence of the Simulated Annealing Algorithm for Continuous Global Optimization

A class of simulated annealing algorithms for continuous global optimization is considered in this paper. The global convergence property is analyzed with respect to the objective value sequence and the minimum objective value sequence induced by simulated annealing algorithms. The convergence analysis provides the appropriate conditions on both the generation probability density function and the temperature updating function. Different forms of temperature updating functions are obtained with respect to different kinds of generation probability density functions, leading to different types of simulated annealing algorithms which all guarantee the convergence to the global optimum.     

基于非拟牛顿方法无约束最优化问题的全局收敛性

In this paper, the non-quasi-Newton's family with inexact line search applied to unconstrained optimization problems is studied. A new update formula for non-quasi-Newton's family is proposed. It is proved that the constituted algorithm with either Wolfe-type or Armijotype line search converges globally and Q-superlinearly if the function to be minimized has Lipschitz continuous gradient.     

Global Optimization: Fractal Approach and Non-redundant Parallelism

More and more optimization problems arising in practice can not be solved by traditional optimization techniques making strong suppositions about the problem (differentiability, convexity, etc.). This happens because very often in real-life problems both the objective function and constraints can be multiextremal, non-differentiable, partially defined, and hard to be evaluated. In this paper, a modern approach for solving such problems (called global optimization problems) is described. This approach combines the following innovative and powerful tools: fractal approach for reduction of the problem dimension, index scheme for treating constraints, non-redundant parallel computations for accelerating the search. Through the paper, rigorous theoretical results are illustrated by figures and numerical examples.     

一种改进的禁忌搜索算法及其在连续全局优化中的应用

禁忌搜索算法是一种元启发式的全局优化算法,是局部搜索算法的一种推广,已被成功地应用于许多组合优化问题中。本文针对有界闭区域上的连续函数全局优化问题,提出了一种改进的禁忌搜索算法,并进行了理论分析和数值实验。数值实验表明,对于连续函数全局优化问题的求解该算法是可行有效的,并且结构简单,迭代次数较少,是一种较好的全局启发式优化算法。     

类Broyden族非拟牛顿算法对一般目标函数的全局收敛性

应用双参数的类Broyden族校正公式,为研究求解无约束最优化问题的拟牛顿类算法对一般目标函数的收敛性这个开问题提供了一种新的方法．     

线性互补问题的异步并行多分裂松弛迭代算法

将求解线性方程组的异步并行多分裂松弛迭代算法推广到线性互补问题.当问题的系数矩阵为H-矩阵类时,证明了算法的全局收敛性.     

The Global Convergence of Self-Scaling BFGS Algorithm with Nonmonotone Line Search for Unconstrained Nonconvex Optimization Problems

The self-scaling quasi-Newton method solves an unconstrained optimization problem by scaling the Hessian approximation matrix before it is updated at each iteration to avoid the possible large eigenvalues in the Hessian approximation matrices of the objective function. It has been proved in the literature that this method has the global and superlinear convergence when the objective function is convex （or even uniformly convex）. We propose to solve unconstrained nonconvex optimization problems by a self-scaling BFGS algorithm with nonmonotone linear search. Nonmonotone line search has been recognized in numerical practices as a competitive approach for solving large-scale nonlinear problems. We consider two different nonmonotone line search forms and study the global convergence of these nonmonotone self-scale BFGS algorithms. We prove that, under some weaker condition than that in the literature, both forms of the self-scaling BFGS algorithm are globally convergent for unconstrained nonconvex optimization problems.     

Global Optimization by Multilevel Coordinate Search

Inspired by a method by Jones et al. (1993), we present a global optimization algorithm based on multilevel coordinate search. It is guaranteed to converge if the function is continuous in the neighborhood of a global minimizer. By starting a local search from certain good points, an improved convergence result is obtained. We discuss implementation details and give some numerical results.     

利用Armijo型线性搜索HZ共轭梯度法的全局收敛性

由William W.Hager和张洪超提出的一种新的共轭梯度法(简称HZ方法),已被证明是一种有效的方法.本文证明了HZ共轭梯度法在Armijo型线性搜索下的全局收敛性.数值实验显示,在Armijo型线性搜索下的HZ共轭梯度法比在Wolfe线性搜索下更有效.     

Wolfe线搜索下一类混合共轭梯度法的全局收敛性

本文给出了一个新的共轭梯度公式,新公式在精确线搜索下与DY公式等价,并给出了新公式的相关性质.结合新公式和DY公式提出了一个新的混合共轭梯度法,新算法在Wolfe线搜索下产生一个下降方向,并证明了算法的全局收敛性,并给出了数值例子.     

Parallel Asynchronous Algorithms for the K Shortest Paths Problem

We deal with the problem of computing in parallel the first K shortest paths from a single origin node to all other nodes of a directed graph. Our approach is based on a very easy way to introduce parallelism in the typical sequential label-correcting method. We describe formally an asynchronous method defined on a shared-memory parallel computational model, and we prove its theoretical correctness under very weak assumptions. According to different strategies used to implement the generic method, we propose several parallel label-correcting algorithms, and we analyze the numerical performance of the resulting codes on a nonuniform memory access multiprocessor via computational results collected on random generated networks. The advantage of parallelism is significant for very large-scale problems of practical interest.