Modified nonmonotone Armijo line search for descent method

Nonmonotone line search approach is a new technique for solving optimization problems. It relaxes the line search range and finds a larger step-size at each iteration, so as to possibly avoid local minimizer and run away from narrow curved valley. It is helpful to find the global minimizer of optimization problems. In this paper we develop a new modification of matrix-free nonmonotone Armijo line search and analyze the global convergence and convergence rate of the resulting method. We also address several approaches to estimate the Lipschitz constant of the gradient of objective functions that would be used in line search algorithms. Numerical results show that this new modification of Armijo line search is efficient for solving large scale unconstrained optimization problems.     

Two improved harmony search algorithms for solving engineering optimization problems

This paper describes two new harmony search (HS) meta-heuristic algorithms for engineering optimization problems with continuous design variables. The key difference between these algorithms and traditional (HS) method is in the way of adjusting bandwidth (bw). bw is very important factor for the high efficiency of the harmony search algorithms and can be potentially useful in adjusting convergence rate of algorithms to optimal solution. First algorithm, proposed harmony search (PHS), introduces a new definition of bandwidth (bw). Second algorithm, improving proposed harmony search (IPHS) employs to enhance accuracy and convergence rate of PHS algorithm. In IPHS, non-uniform mutation operation is introduced which is combination of Yang bandwidth and PHS bandwidth. Various engineering optimization problems, including mathematical function minimization problems and structural engineering optimization problems, are presented to demonstrate the effectiveness and robustness of these algorithms. In all cases, the solutions obtained using IPHS are in agreement or better than those obtained from other methods.     

Mesh adaptive direct search algorithms for mixed variable optimization

This paper introduces a new derivative-free class of mesh adaptive direct search (MADS) algorithms for solving constrained mixed variable optimization problems, in which the variables may be continuous or categorical. This new class of algorithms, called mixed variable MADS (MV-MADS), generalizes both mixed variable pattern search (MVPS) algorithms for linearly constrained mixed variable problems and MADS algorithms for general constrained problems with only continuous variables. The convergence analysis, which makes use of the Clarke nonsmooth calculus, similarly generalizes the existing theory for both MVPS and MADS algorithms, and reasonable conditions are established for ensuring convergence of a subsequence of iterates to a suitably defined stationary point in the nonsmooth and mixed variable sense.     

Retrospective optimization of mixed-integer stochastic systems using dynamic simplex linear interpolation

We propose a family of retrospective optimization (RO) algorithms for optimizing stochastic systems with both integer and continuous decision variables. The algorithms are continuous search procedures embedded in a RO framework using dynamic simplex interpolation (RODSI). By decreasing dimensions (corresponding to the continuous variables) of simplex, the retrospective solutions become closer to an optimizer of the objective function. We present convergence results of RODSI algorithms for stochastic “convex” systems. Numerical results show that a simple implementation of RODSI algorithms significantly outperforms some random search algorithms such as Genetic Algorithm (GA) and Particle Swarm Optimization (PSO).     

Analyzing the Performance of Generalized Hill Climbing Algorithms

Generalized hill climbing algorithms provide a framework to describe and analyze metaheuristics for addressing intractable discrete optimization problems. The performance of such algorithms can be assessed asymptotically, either through convergence results or by comparison to other algorithms. This paper presents necessary and sufficient convergence conditions for generalized hill climbing algorithms. These conditions are shown to be equivalent to necessary and sufficient convergence conditions for simulated annealing when the generalized hill climbing algorithm is restricted to simulated annealing. Performance measures are also introduced that permit generalized hill climbing algorithms to be compared using random restart local search. These results identify a solution landscape parameter based on the basins of attraction for local optima that determines whether simulated annealing or random restart local search is more effective in visiting a global optimum. The implications and limitations of these results are discussed.     

Convergence guarantees for generalized adaptive stochastic search methods for continuous global optimization

This paper presents some simple technical conditions that guarantee the convergence of a general class of adaptive stochastic global optimization algorithms. By imposing some conditions on the probability distributions that generate the iterates, these stochastic algorithms can be shown to converge to the global optimum in a probabilistic sense. These results also apply to global optimization algorithms that combine local and global stochastic search strategies and also those algorithms that combine deterministic and stochastic search strategies. This makes the results applicable to a wide range of global optimization algorithms that are useful in practice. Moreover, this paper provides convergence conditions involving the conditional densities of the random vector iterates that are easy to verify in practice. It also provides some convergence conditions in the special case when the iterates are generated by elliptical distributions such as the multivariate Normal and Cauchy distributions. These results are then used to prove the convergence of some practical stochastic global optimization algorithms, including an evolutionary programming algorithm. In addition, this paper introduces the notion of a stochastic algorithm being probabilistically dense in the domain of the function and shows that, under simple assumptions, this is equivalent to seeing any point in the domain with probability 1. This, in turn, is equivalent to almost sure convergence to the global minimum. Finally, some simple results on convergence rates are also proved.     

一类带非精确线搜索的修改的Broyden算法

对于文（８）和（１４）中提出的修改的Ｂｒｏｙｄｅｎ算法，本文讨论它在线搜索非精确时的收敛性质，证明这类算法作用于梯度满足Ｌｉｐｓｃｈｉｔｚ条件的目标函数时是整体收敛的，当目标函数一致凸时，算法是Ｑ－超线性收敛和二阶收敛的。     

一簇非线性等式约束优化问题的过滤线搜索修正正割方法

本文提供了一簇新的过滤线搜索修正正割方法求解非线性等式约束优化问题.新算法簇的特点是:用修正正割算法簇中的一个算法获得搜索方向,回代线搜索技术得到步长,过滤准则用来决定是否接受步长,引入二阶校正技术减少不可行性并克服Maratos效应.在合理的假设条件下,分析了算法的总体收敛性.并证明了,通过附加二阶校正步,算法簇克服了Maratos效应,并二步Q-超线性收敛到满足二阶充分最优条件的局部解.数值结果表明了所提供的算法具有有效性.     

A Class of Revised Broyden Algorithms Without Exact Line Search

In this paper, we discuss the convergence of the Broyden algorithms with revised search direction. Under some inexact line searches, we prove that the algorithms are globally convergent for continuously differentiable functions and the rate of local convergence of the algorithms is one-step superlinear and n-step second-order for uniformly convex objective functions.     

Global Convergence of Algorithms with Nonmonotone Line Search Strategy in Unconstrained Optimization

In this paper we state some nonmonotone line search strategies for unconstrained optimization algorithms. Abstracting from the concrete line search strategy we prove two general convergence results. Using this theory we can show the global convergence of the BFGS method with nonmonotone line search strategy. In contrast to some former results about nonmonotone line search strategies, both our convergence results and their proofs are natural generalizations of known results for the monotone case.     

基于梯度的自适应快速布谷鸟搜索算法

针对标准布谷鸟搜索(CS)算法存在全局搜索和局部搜索能力不平衡的缺点, 提出一种基于梯度的自适应快速布谷鸟搜索(GBAQCS)算法. 在改进的算法中, 针对偏好随机游动的步长, 在利用目标函数的梯度决定步长方向的基础上, 首先提出自适应搜索机制平衡了算法的全局搜索和局部搜索能力; 其次提出快速 搜索策略, 充分利用当前鸟巢信息进行精细化搜索, 从而提高算法的搜索精度和收敛速度. 实验结果表明, 相比其他算法, 所提出的改进策略使算法的全局搜索和局部搜索能力保持了相对的平衡, 并提高了算法的收敛性能.     

带非精确线搜索的调整搜索方向DFP算法

本文介绍一类新的带调整搜索方向的Broyden算法．我们着重讨论带调整搜索方向的DFP算法的收敛性，在某些非精确线搜索的情况下，我们证明对连续可微目标函数，这算法是整体收敛的，而对一致凸目标函数，收敛速度是一步超线收敛的．从这篇文章的证明过程中，可以得到对一致凸目标函数，DFP算法具有一步超线形收敛．     

Two minimal positive bases based direct search conjugate gradient methods for computationally expensive functions

Positive basis is an important concept in direct search methods. Although any positive basis can ensure the convergence in theory, the maximum positive bases are often used to construct direct search algorithms. In this paper, two direct search methods for computational expensive functions are proposed based on the minimal positive bases. The Coope–Price’s frame-based direct search framework is employed to insure convergence. PRP+ method and a recently developed descent conjugate gradient method are employed respectively to accelerate convergence. The data profiles and the performance profiles of the numerical experiments show that the proposed methods are effective for computational expensive functions.     

Numerical experiments on dual matrix algorithms for function minimization

In Ref. 2, four algorithms of dual matrices for function minimization were introduced. These algorithms are characterized by the simultaneous use of two matrices and by the property that the one-dimensional search for the optimal stepsize is not needed for convergence. For a quadratic function, these algorithms lead to the solution in at mostn+1 iterations, wheren is the number of variables in the function. Since the one-dimensional search is not needed, the total number of gradient evaluations for convergence is at mostn+2. In this paper, the above-mentioned algorithms are tested numerically by using five nonquadratic functions. In order to investigate the effects of the stepsize on the performances of these algorithms, four schemes for the stepsize factor are employed, two corresponding to small-step processes and two corresponding to large-step processes. The numerical results show that, in spite of the wide range employed in the choice of the stepsize factor, all algorithms exhibit satisfactory convergence properties and compare favorably with the corresponding quadratically convergent algorithms using one-dimensional searches for optimal stepsizes.     

A Theoretical and Numerical Comparison of Some Semismooth Algorithms for Complementarity Problems

In this paper we introduce a general line search scheme which easily allows us to define and analyze known and new semismooth algorithms for the solution of nonlinear complementarity problems. We enucleate the basic assumptions that a search direction to be used in the general scheme has to enjoy in order to guarantee global convergence, local superlinear/quadratic convergence or finite convergence. We examine in detail several different semismooth algorithms and compare their theoretical features and their practical behavior on a set of large-scale problems.     

On the rate of convergence of one inhomogeneous Markov algorithm of search for extremum

The paper is concerned with theoretical study of the rate of convergence of one inhomogeneous Markov random algorithm of search for extremum. Methods of random search have value in solving involved optimization problems. However, there are only few theoretical results concerning the rate of convergence of these algorithms.     

Global Optimization using Dynamic Search Trajectories

Two global optimization algorithms are presented. Both algorithms attempt to minimize an unconstrained objective function through the modeling of dynamic search trajectories. The first, namely the Snyman–Fatti algorithm, originated in the 1980's and still appears an effective global optimization algorithm. The second algorithm is currently under development, and is denoted the modified bouncing ball algorithm. For both algorithms, the search trajectories are modified to increase the likelihood of convergence to a low local minimum. Numerical results illustrate the effectiveness of both algorithms.     

Global Optimization Performance Measures for Generalized Hill Climbing Algorithms

Generalized hill climbing algorithms provide a framework for modeling several local search algorithms for hard discrete optimization problems. This paper introduces and analyzes generalized hill climbing algorithm performance measures that reflect how effectively an algorithm has performed to date in visiting a global optimum and how effectively an algorithm may pes]rform in the future in visiting such a solution. These measures are also used to obtain a necessary asymptotic convergence (in probability) condition to a global optimum, which is then used to show that a common formulation of threshold accepting does not converge. These measures assume particularly simple forms when applied to specific search strategies such as Monte Carlo search and threshold accepting.     

一类超线性收敛的广义拟Newton算法

１引言考虑无约束最优化问题其中目标函数ｆ（ｘ）二阶连续可微,记ｆｋ＝ｆ（ｘ）,当充分小时,有如下近似关系：它们对二次函数皆严格成立．考虑选代其中Ｂ（Ｇ的近似）已知,为某种线搜索确定的步长．对Ｂ修正产生Ｂ,即Ｕ为待定ｎ阶矩阵．若要求Ｂ＋满足关系即Ｂ满足拟Ｎｅｗｔｏｎ方程,由它可导出许多著名的拟Ｎｅｗｔｏｎ算法［１－［４］）．若要求Ｂ满足关系则可导出伪Ｎｅｗｔｏｎ－δ族校正公式,它不再是Ｈｕａｎｇ族成员［６］．从信息资源的利用看,（１．６）仅利用了与信息,（１．７）仅利用了与信息．一般而言,较多的信…     

Asynchronous parallel successive overrelaxation for the symmetric linear complementarity problem

Convergence is established for asynchronous parallel successive overrelaxation (SOR) algorithms for the symmetric linear complementarity problem. For the case of a strictly diagonally dominant matrix convergence is achieved for a relaxation factor interval of (0, 2] with line search, and (0, 1] without line search. Computational tests on the Sequent Symmetry S81 multiprocessor give speedup efficiency in the 43%–91% range for the cases for which convergence is established. The tests also show superiority of the asynchronous SOR algorithms over their synchronous counterparts.This material is based on research supported by National Science Foundation Grants DCR-8420963 and DCR-8521228 and Air Force Office of Scientific Research Grant AFOSR-86-0172.