Descent Direction Stochastic Approximation Algorithm with Adaptive Step Sizes

A stochastic approximation (SA) algorithm with new adaptive step sizes for solving unconstrained minimization problems in noisy environment is proposed. New adaptive step size scheme uses ordered statistics of fixed number of previous noisy function values as a criterion for accepting good and rejecting bad steps. The scheme allows the algorithm to move in bigger steps and avoid steps proportional to $1/k$ when it is expected that larger steps will improve the performance. An algorithm with the new adaptive scheme is defined for a general descent direction. The almost sure convergence is established. The performance of new algorithm is tested on a set of standard test problems and compared with relevant algorithms. Numerical results support theoretical expectations and verify efficiency of the algorithm regardless of chosen search direction and noise level. Numerical results on problems arising in machine learning are also presented. Linear regression problem is considered using real data set. The results suggest that the proposed algorithm shows promise.     

A Stochastic Approximation Algorithm for American Lookback Put Options

This work is concerned with pricing American fixed lookback put options. The underlying asset is modeled as a switching diffusion process, where the switching is represented by a continuous-time Markov chain. The switching diffusion delineates stochastic volatility effectively. Nevertheless, this formulation together with the lookback style put option makes it virtually impossible to find closed-form solutions. As a viable alternative, a stochastic approximation algorithm is suggested. The convergence and rates of convergence of the algorithm are established.     

Stochastic Approximation Algorithms for Parameter Estimation in Option Pricing with Regime Switching

Abstract

This work is concerned with option pricing. Stochastic approximation/optimization algorithms are proposed and analyzed. The underlying stock price evolves according to two geometric Brownian motions coupled by a continuous-time finite state Markov chain. Recursive stochastic approximation algorithms are developed to estimate the implied volatility. Convergence of the algorithm is proved. Rate of convergence is also ascertained. Then real market data are used to compare our algorithms with other schemes.     

Adaptive Algorithms of Nonlinear Approximation with Finite Terms

This paper deals with realizable adaptive algorithms of the nonlinear approximation with finite terms based on wavelets. We present a concrete algorithm by which we may find the required index set Am for the greedy algorithm Gm^P（., Ψ）. This makes the greedy algorithm realize the near best approximation in practice. Moreover, we study the efficiency of the finite-term approximation of another Mgorithm introduced by Birge and Massart.     

A Strong Approximation Theorem for Stochastic Recursive Algorithms

The constant stepsize analog of Gelfand–Mitter type discrete-time stochastic recursive algorithms is shown to track an associated stochastic differential equation in the strong sense, i.e., with respect to an appropriate divergence measure.     

Adaptive Monte Carlo Variance Reduction for Lévy Processes with Two-Time-Scale Stochastic Approximation

We propose an approach to a twofold optimal parameter search for a combined variance reduction technique of the control variates and the important sampling in a suitable pure-jump Lévy process framework. The parameter search procedure is based on the two-time-scale stochastic approximation algorithm with equilibrated control variates component and with quasi-static importance sampling one. We prove the almost sure convergence of the algorithm to a unique optimum. The parameter search algorithm is further embedded in adaptive Monte Carlo simulations in the case of the gamma distribution and process. Numerical examples of the CDO tranche pricing with the Gamma copula model and the intensity Gamma model are provided to illustrate the effectiveness of our method.      

A Stochastic Approximation Method and Its Application to Confidence Intervals

The oldest stochastic approximation method is the Robbins–Monro process. This estimates an unknown scalar parameter by stepping from one trial value for the parameter to another, adopting the last trial value as the estimate. More recent research suggests there are benefits from taking larger steps than with the Robbins–Monro process and then obtaining an estimate by averaging the later trial values. Work on the averaged estimator has made only general assumptions and here we consider a more explicit case that is of practical importance. Stronger asymptotic results are developed and simulations show they hold well for moderately long searches. The results motivate the development of a new method of searching for the endpoints of a confidence interval. This method performs decidedly better than a previously proposed method in terms of both the position of endpoints and the coverage of confidence intervals. The efficiency of the new method is typically well in excess of 90%.

Computer code and an appendix containing the derivation of asymptotic efficiencies are available in the online supplements.     

Strong Predictor-Corrector Approximation for Stochastic Delay Differential Equations

This paper presents a strong predictor-corrector method for the numerical solution of stochastic delay differential equations (SDDEs) of Itô-type. The method is proved to be mean-square convergent of order min{$1/2, \hat{p}$} under the Lipschitz condition and the linear growth condition, where $\hat{p}$ is the exponent of Hölder condition of the initial function. Stability criteria for this type of method are derived. It is shown that for certain choices of the flexible parameter $p$ the derived method can have a better stability property than more commonly used numerical methods. That is, for some $p$, the asymptotic MS-stability bound of the method will be much larger than that of the Euler-Maruyama method. Numerical results are reported confirming convergence properties and comparing stability properties of methods with different parameters $p$. Finally, the vectorised simulation is discussed and it is shown that this implementation is much more efficient.     

Meshfree Approximation for Stochastic Optimal Control Problems

In this work,we study the gradient projection method for solving a class of stochastic control problems by using a mesh free approximation ap-proach to implement spatial dimension approximation.Our main contribu-tion is to extend the existing gradient projection method to moderate high-dimensional space.The moving least square method and the general radial basis function interpolation method are introduced as showcase methods to demonstrate our computational framework,and rigorous numerical analysis is provided to prove the convergence of our meshfree approximation approach.We also present several numerical experiments to validate the theoretical re-sults of our approach and demonstrate the performance meshfree approxima-tion in solving stochastic optimal control problems.     

Adaptive Bivariate Chebyshev Approximation

We propose an adaptive algorithm which extends Chebyshev series approximation to bivariate functions, on domains which are smooth transformations of a square. The method is tested on functions with different degrees of regularity and on domains with various geometries. We show also an application to the fast evaluation of linear and nonlinear bivariate integral transforms.     

任意随机序列关于二重非齐次马氏链的随机和的一类随机选择系统的随机逼近定理

本文引进任意随机变量序列随机极限对数似然比的概念,通过测度$\pr$下任意相依随机序列联合分布与测度$\qr$下二重非齐次马氏分布相比较,利用母函数与尾概率母函数工具研究任意受控随机序列之随机和在随机选择系统中的一类随机逼近定理.     

Rooted Tree Analysis for Order Conditions of Stochastic Runge-Kutta Methods for the Weak Approximation of Stochastic Differential Equations

Abstract

A general class of stochastic Runge-Kutta methods for the weak approximation of Itô and Stratonovich stochastic differential equations with a multi-dimensional Wiener process is introduced. Colored rooted trees are used to derive an expansion of the solution process and of the approximation process calculated with the stochastic Runge-Kutta method. A theorem on general order conditions for the coefficients and the random variables of the stochastic Runge-Kutta method is proved by rooted tree analysis. This theorem can be applied for the derivation of stochastic Runge-Kutta methods converging with an arbitrarily high order.     

求解log—最优组合投资问题的一个自适应算法及其理论分析

本文给出了一个求解log－最优组合投资问题的自适应算法,它是一个变型的随机逼近方法。该问题是一个约束优化问题,因此,采用基于约束流形的梯度上升方向替代常规梯度上升方向,在一些合理的假设下证明了算法的收敛性并进行了渐近稳定性分析。最后,本文将该算法应用于上海证券交易所提供的实际数据的log-最优组合投资问题求解,获得了理想的数值模拟结果。     

Regression Density Estimation With Variational Methods and Stochastic Approximation

Regression density estimation is the problem of flexibly estimating a response distribution as a function of covariates. An important approach to regression density estimation uses finite mixture models and our article considers flexible mixtures of heteroscedastic regression (MHR) models where the response distribution is a normal mixture, with the component means, variances, and mixture weights all varying as a function of covariates. Our article develops fast variational approximation (VA) methods for inference. Our motivation is that alternative computationally intensive Markov chain Monte Carlo (MCMC) methods for fitting mixture models are difficult to apply when it is desired to fit models repeatedly in exploratory analysis and model choice. Our article makes three contributions. First, a VA for MHR models is described where the variational lower bound is in closed form. Second, the basic approximation can be improved by using stochastic approximation (SA) methods to perturb the initial solution to attain higher accuracy. Third, the advantages of our approach for model choice and evaluation compared with MCMC-based approaches are illustrated. These advantages are particularly compelling for time series data where repeated refitting for one-step-ahead prediction in model choice and diagnostics and in rolling-window computations is very common. Supplementary materials for the article are available online.     

随机容错设施布局问题的近似算法

在确定性的容错设施布局问题中, 给定顾客的集合和地址的集合. 在每个地址上可以开设任意数目的不同设施. 每个顾客j有连接需求r_j. 允许将顾客j连到同一地址的不同设施上. 目标是开设一些设施并将每个顾客j连到r_j个不同的设施上, 使得总开设费用和连接费用最小. 研究两阶段随机容错设施布局问题(SFTFP), 顾客的集合事先不知道, 但是具有有限多个场景并知道其概率分布. 每个场景指定需要服务的顾客的子集. 并且每个设施有两种类型的开设费用. 在第一阶段根据顾客的随机信息确定性地开设一些设施, 在第二阶段根据顾客的真实信息再增加开设一些设施．给出随机容错布局问题的线性整数规划和基于线性规划舍入的5-近似算法.     

Gaussian Variational Approximation With a Factor Covariance Structure

Variational approximations have the potential to scale Bayesian computations to large datasets and highly parameterized models. Gaussian approximations are popular, but can be computationally burdensome when an unrestricted covariance matrix is employed and the dimension of the model parameter is high. To circumvent this problem, we consider a factor covariance structure as a parsimonious representation. General stochastic gradient ascent methods are described for efficient implementation, with gradient estimates obtained using the so-called “reparameterization trick.” The end result is a flexible and efficient approach to high-dimensional Gaussian variational approximation. We illustrate using robust P-spline regression and logistic regression models. For the latter, we consider eight real datasets, including datasets with many more covariates than observations, and another with mixed effects. In all cases, our variational method provides fast and accurate estimates. Supplementary material for this article is available online.     

Biconjugate directions

It is shown that the generalization of the conjugate direction method of Van Wyk (Ref. 1) is the direction counterpart to Fletcher's biconjugate gradient algorithm (Ref. 2).     

Blind Channel Identification Based on Noisy Observation by Stochastic Approximation Method

A stochastic approximation algorithm for estimating multichannel coefficients is proposed, and the estimate is proved to converge to the true parameters a.s. up-to a constant scaling factor. The estimate is updated after receiving each new observation, so the output data need not be collected in advance. The input signal is allowed to be dependent and the observation is allowed to be corrupted by noise, but no noise statistics are used in the estimation algorithm.     

关于圆和椭圆逼近显示的最优算法(英文)

本文给出了一个逼近显示圆的新算法。该算法是通过相交多边形而不是内接多边形逼近圆。由于构造相交多边形时其面积等于圆面积 ,因此新算法是最优逼近。同时还推广到椭圆