| A Stochastic Approximation Frame Algorithm with Adaptive Directions |
| |
| 作者单位： | Zi Xu~1 Yu-Hong Dai~2 1 Department of Mathematics，College of Sciences，Shanghai University，Shanghai，200444，P.R.China. 2 State Key Laboratory of Scientific and Engineering Computing，Institute of Computational Mathematics and Scientific/Engineering Computing，The Academy of Mathematics and Systems Science，Chinese Academy of Sciences，P.O.Box 2719，Beijing，100190，P.R.China. |
| |
| 摘 要： | Stochastic approximation problem is to find some root or extremum of a non- linear function for which only noisy measurements of the function are available.The classical algorithm for stochastic approximation problem is the Robbins-Monro (RM) algorithm,which uses the noisy evaluation of the negative gradient direction as the iterative direction.In order to accelerate the RM algorithm,this paper gives a flame algorithm using adaptive iterative directions.At each iteration,the new algorithm goes towards either the noisy evaluation of the negative gradient direction or some other directions under some switch criterions.Two feasible choices of the criterions are pro- posed and two corresponding flame algorithms are formed.Different choices of the directions under the same given switch criterion in the flame can also form different algorithms.We also proposed the simultanous perturbation difference forms for the two flame algorithms.The almost surely convergence of the new algorithms are all established.The numerical experiments show that the new algorithms are promising.
|
| 关 键 词： | 随机逼近 共轭梯度 自适应方向 极值 |
A Stochastic Approximation Frame Algorithm with Adaptive Directions |
| |
| Authors: | Zi Xu YuHong Dai |
| |
| Institution: | 1. Department of Mathematics, College of Sciences, Shanghai University, Shang-hai, 200444, P.R.China
2. State Key Laboratory of Scientific and Engineering Computing,Institute of Computational Mathematics and Scientific/Engineering Computing,The Academy of Mathematics and System Systems Science,Chinese Academy of Sciences P.O.Box2719 Beijing,100190,P.R.China |
| |
| Abstract: | Stochastic approximation problem is to find some root or extremum of a non- linear function for which only noisy measurements of the function are available.The classical algorithm for stochastic approximation problem is the Robbins-Monro (RM) algorithm,which uses the noisy evaluation of the negative gradient direction as the iterative direction.In order to accelerate the RM algorithm,this paper gives a flame algorithm using adaptive iterative directions.At each iteration,the new algorithm goes towards either the noisy evaluation of the negative gradient direction or some other directions under some switch criterions.Two feasible choices of the criterions are pro- posed and two corresponding flame algorithms are formed.Different choices of the directions under the same given switch criterion in the flame can also form different algorithms.We also proposed the simultanous perturbation difference forms for the two flame algorithms.The almost surely convergence of the new algorithms are all established.The numerical experiments show that the new algorithms are promising. |
| |
| Keywords: | Stochastic approximation conjugate gradient adaptive directions |
| 本文献已被 CNKI 维普 万方数据 等数据库收录！ |
