A Stochastic Approximation Frame Algorithm with Adaptive Directions

Stochastic approximation problem is to find some root or extremum of a non- linear function for which only noisy measurements of the function are available.The classical algorithm for stochastic approximation problem is the Robbins-Monro (RM) algorithm,which uses the noisy evaluation of the negative gradient direction as the iterative direction.In order to accelerate the RM algorithm,this paper gives a flame algorithm using adaptive iterative directions.At each iteration,the new algorithm goes towards either the noisy evaluation of the negative gradient direction or some other directions under some switch criterions.Two feasible choices of the criterions are pro- posed and two corresponding flame algorithms are formed.Different choices of the directions under the same given switch criterion in the flame can also form different algorithms.We also proposed the simultanous perturbation difference forms for the two flame algorithms.The almost surely convergence of the new algorithms are all established.The numerical experiments show that the new algorithms are promising.     

Efficient Nonnegative Matrix Factorization Via Modified Monotone Barzilai-Borwein Method with Adaptive Step Sizes Strategy

In this paper, we develop an active set identification technique. By means of the active set technique, we present an active set adaptive monotone projected Barzilai-Borwein method (ASAMPBB) for solving nonnegative matrix factorization (NMF) based on the alternating nonnegative least squares framework, in which the Barzilai-Borwein (BB) step sizes can be adaptively picked to get meaningful convergence rate improvements. To get optimal step size, we take into account of the curvature information. In addition, the larger step size technique is exploited to accelerate convergence of the proposed method. The global convergence of the proposed method is analysed under mild assumption. Finally, the results of the numerical experiments on both synthetic and real-world datasets show that the proposed method is effective.     

带下降方向搜索的总极值方法

本文将文[1]提出的一类求总极值方法与非线性规划的下降方法相结合,提出一类带下降方向搜索的求总极值方法,并且证明了这类方法具有线性和超线性收效率.     

一个基于近似下降方向的免梯度算法

本构造一个求解非线性无约束优化问题的免梯度算法，该算法基于传统的模矢法，每次不成功迭代后，充分利用已有迭代点的信息，构造近似下降方向，产生新的迭代点。在较弱条件下，算法是总体收敛的。通过数值实验与传统模矢法相比，计算量明显减少。     

带有固定步长的非单调自适应信赖域算法

提出了求解无约束优化问题带有固定步长的非单调自适应信赖域算法.信赖域半径的修正采用自适应技术,算法在试探步不被接受时,采用固定步长寻找下一迭代点.并在适当的条件下,证明算法具有全局收敛性和超线性收敛性.初步的数值试验表明算法对高维问题具有较好的效果.     

Adaptive Stabilization of Nonlinear Stochastic Systems

The purpose of this paper is to study the problem of asymptotic stabilization in probability of nonlinear stochastic differential systems with unknown parameters. With this aim, we introduce the concept of an adaptive control Lyapunov function for stochastic systems and we use the stochastic version of Artstein's theorem to design an adaptive stabilizer. In this framework the problem of adaptive stabilization of a nonlinear stochastic system is reduced to the problem of asymptotic stabilization in probability of a modified system. The design of an adaptive control Lyapunov function is illustrated by the example of adaptively quadratically stabilizable in probability stochastic differential systems. Accepted 9 December 1996     

An Adaptive Version for the Metropolis Adjusted Langevin Algorithm with a Truncated Drift

This paper extends some adaptive schemes that have been developed for the Random Walk Metropolis algorithm to more general versions of the Metropolis-Hastings (MH) algorithm, particularly to the Metropolis Adjusted Langevin algorithm of Roberts and Tweedie (1996). Our simulations show that the adaptation drastically improves the performance of such MH algorithms. We study the convergence of the algorithm. Our proves are based on a new approach to the analysis of stochastic approximation algorithms based on mixingales theory.      

Adaptive Monte Carlo Variance Reduction for Lévy Processes with Two-Time-Scale Stochastic Approximation

We propose an approach to a twofold optimal parameter search for a combined variance reduction technique of the control variates and the important sampling in a suitable pure-jump Lévy process framework. The parameter search procedure is based on the two-time-scale stochastic approximation algorithm with equilibrated control variates component and with quasi-static importance sampling one. We prove the almost sure convergence of the algorithm to a unique optimum. The parameter search algorithm is further embedded in adaptive Monte Carlo simulations in the case of the gamma distribution and process. Numerical examples of the CDO tranche pricing with the Gamma copula model and the intensity Gamma model are provided to illustrate the effectiveness of our method.