Stochastic approximation and the final value theorem

The aim here is to show how to obtain many of the well-known limit results (i.e., central limit theorem, law of the iterated logarithm, invariance principle) of stochastic approximation (SA) by a shorter argument and under weaker conditions. The idea is to introduce an artificial sequence, related to the SA scheme, and which clearly obeys the limit law. This sequence is subtracted from the SA scheme and then simple deterministic limit theory is used to show the remainder is negligible. As a consequence of this approach proofs are shorter and the meaning of conditions becomes clearer. Because the difference equations are not summed up it is simple to state results for general a_n, c_n sequences.     

Stochastic approximation algorithm for minimax problems

A stochastic approximation algorithm for minimax optimization problems is analyzed. At each iterate, it performs one random experiment, based on which it computes a direction vector. It is shown that, under suitable conditions, it a.s. converges to the set of points satisfying necessary optimality conditions. The algorithm and its analysis bring together ideas from stochastic approximation and nondifferentiable optimization.     

Stochastic approximation with long range dependent and heavy tailed noise

Stability and convergence properties of stochastic approximation algorithms are analyzed when the noise includes a long range dependent component (modeled by a fractional Brownian motion) and a heavy tailed component (modeled by a symmetric stable process), in addition to the usual ‘martingale noise’. This is motivated by the emergent applications in communications. The proofs are based on comparing suitably interpolated iterates with a limiting ordinary differential equation. Related issues such as asynchronous implementations, Markov noise, etc. are briefly discussed.     

Stochastic recursive inclusions with non-additive iterate-dependent Markov noise

In this paper we study the asymptotic behaviour of stochastic approximation schemes with set-valued drift function and non-additive iterate-dependent Markov noise. We show that a linearly interpolated trajectory of such a recursion is an asymptotic pseudotrajectory for the flow of a limiting differential inclusion obtained by averaging the set-valued drift function of the recursion w.r.t. the stationary distributions of the Markov noise. The limit set theorem by Benaim is then used to characterize the limit sets of the recursion in terms of the dynamics of the limiting differential inclusion. We then state two variants of the Markov noise assumption under which the analysis of the recursion is similar to the one presented in this paper. Scenarios where our recursion naturally appears are presented as applications. These include controlled stochastic approximation, subgradient descent, approximate drift problem and analysis of discontinuous dynamics all in the presence of non-additive iterate-dependent Markov noise.     

Invariance principles and Gaussian approximation for strictly stationary processes

We show that in any aperiodic and ergodic dynamical system there exists a square integrable process the partial sums of which can be closely approximated by the partial sums of Gaussian i.i.d. random variables. For both weak and strong invariance principles hold.

     

Arithmetic means and invariance principles in stochastic approximation

Summation for the sequence (U _n ) _n N of random variables with values in a real separable Banach space recursively defined by

Meshfree Approximation for Stochastic Optimal Control Problems

In this work,we study the gradient projection method for solving a class of stochastic control problems by using a mesh free approximation ap-proach to implement spatial dimension approximation.Our main contribu-tion is to extend the existing gradient projection method to moderate high-dimensional space.The moving least square method and the general radial basis function interpolation method are introduced as showcase methods to demonstrate our computational framework,and rigorous numerical analysis is provided to prove the convergence of our meshfree approximation approach.We also present several numerical experiments to validate the theoretical re-sults of our approach and demonstrate the performance meshfree approxima-tion in solving stochastic optimal control problems.     

Stochastic Hopf bifurcation and chaos of stochastic Bonhoeffer–van der Pol system via Chebyshev polynomial approximation

In this paper, we analyzed stochastic chaos and Hopf bifurcation of stochastic Bonhoeffer–van der Pol (SBVP for short) system with bounded random parameter of an arch-like probability density function. The modifier ‘stochastic’ here implies dependent on some random parameter. In order to study the dynamical behavior of the SBVP system, Chebyshev polynomial approximation is applied to transform the SBVP system into its equivalent deterministic system, whose response can be readily obtained by conventional numerical methods. Thus, we can further explore the nonlinear phenomena in SBVP system. Stochastic chaos and Hopf bifurcation analyzed here are by and large similar to those in the deterministic mean-parameter Bonhoeffer–van der Pol system (DM–BVP for short) but there are also some featuring differences between them shown by numerical results. For example, in the SBVP system the parameter interval matching chaotic responses diffuses into a wider one, which further grows wider with increasing of intensity of the random variable. The shapes of limit cycles in the SBVP system are some different from that in the DM–BVP system, and the sizes of limit cycles become smaller with the increasing of intensity of the random variable. And some biological explanations are given.     

Pathwise convergence rates for numerical solutions of Markovian switching stochastic differential equations

This work develops numerical approximation algorithms for solutions of stochastic differential equations with Markovian switching. The existing numerical algorithms all use a discrete-time Markov chain for the approximation of the continuous-time Markov chain. In contrast, we generate the continuous-time Markov chain directly, and then use its skeleton process in the approximation algorithm. Focusing on weak approximation, we take a re-embedding approach, and define the approximation and the solution to the switching stochastic differential equation on the same space. In our approximation, we use a sequence of independent and identically distributed (i.i.d.) random variables in lieu of the common practice of using Brownian increments. By virtue of the strong invariance principle, we ascertain rates of convergence in the pathwise sense for the weak approximation scheme.     

Stochastic adaptive control with small observation noise

Controlled diffusions depending on an unknown parameter and with small system perturbation are considered in this paper. Two parameter identification methods are proposed and error probabilities are estimated in terms of the small perturbation parameter. These methods are then used to choose among competing filters on successive time intervals. Asymptotically optimal controls based on partial observations are found on successive time intervals by using the best filter identified on the previous time intervals.     

Maximum Principle for Stochastic Differential Games with Partial Information

In this paper, we first deal with the problem of optimal control for zero-sum stochastic differential games. We give a necessary and sufficient maximum principle for that problem with partial information. Then, we use the result to solve a problem in finance. Finally, we extend our approach to general stochastic games (nonzero-sum), and obtain an equilibrium point of such game.     

A Stochastic Approximation Frame Algorithm with Adaptive Directions

Stochastic approximation problem is to find some root or extremum of a non- linear function for which only noisy measurements of the function are available.The classical algorithm for stochastic approximation problem is the Robbins-Monro (RM) algorithm,which uses the noisy evaluation of the negative gradient direction as the iterative direction.In order to accelerate the RM algorithm,this paper gives a flame algorithm using adaptive iterative directions.At each iteration,the new algorithm goes towards either the noisy evaluation of the negative gradient direction or some other directions under some switch criterions.Two feasible choices of the criterions are pro- posed and two corresponding flame algorithms are formed.Different choices of the directions under the same given switch criterion in the flame can also form different algorithms.We also proposed the simultanous perturbation difference forms for the two flame algorithms.The almost surely convergence of the new algorithms are all established.The numerical experiments show that the new algorithms are promising.     

Stochastic optimization with averaging of trajectories

A recursive stochastic optimization procedure under dependent disturbances is studied. It is based on the Polyak-Ruppert algorithm with trajectory averaging. Almost sure convergence of the algorithm is proved as well as asymptotic normality of the delivered estimates. It is shown that the presented algorithm attains the highest possible asymptotic convergence rate for stochastic approximation algorithms     

Stochastic uniform observability of linear differential equations with multiplicative noise

The aim of this paper is to give a deterministic characterization of the uniform observability property of linear differential equations with multiplicative white noise in infinite dimensions. We also investigate the properties of a class of perturbed evolution operators and we used these properties to give a new representation of the covariance operators associated to the mild solutions of the investigated stochastic differential equations. The obtained results play an important role in obtaining necessary and sufficient conditions for the stochastic uniform observability property.     

Finite Difference Approximations for Stochastic Control Systems with Delay

Abstract

This article considers the computation issues of the infinite dimensional HJB equation arising from the finite horizon optimal control problem of a general system of stochastic functional differential equations with a bounded memory treated in [2 Chang , M.H. , Pang , T. , and Pemy , M. accepted. Optimal control of functional stochastic differential equations with a bounded memory. Stochastics 80 ( 1 ): 69 – 96 . [Google Scholar]]. The finite difference scheme, using the result in [1 Barles , G. , and Souganidis , P.E. 1991 . Convergence of approximative schemes for fully nonlinear second order equations . J. Asymptotic Analysis 4 : 557 – 579 . [Google Scholar]], is obtained to approximate the viscosity solution of the infinite dimensional HJB equation. The convergence of the scheme is proved using the Banach fixed point theorem. The computational algorithm also is provided based on the scheme obtained.     

Interpolation and approximation in

Assume a standard Brownian motion W=(W_t)_t[0,1], a Borel function such that f(W₁)L₂, and the standard Gaussian measure γ on the real line. We characterize that f belongs to the Besov space , obtained via the real interpolation method, by the behavior of , where is a deterministic time net and the orthogonal projection onto a subspace of ‘discrete’ stochastic integrals with X being the Brownian motion or the geometric Brownian motion. By using Hermite polynomial expansions the problem is reduced to a deterministic one. The approximation numbers a_X(f(X₁);τ) can be used to describe the L₂-error in discrete time simulations of the martingale generated by f(W₁) and (in stochastic finance) to describe the minimal quadratic hedging error of certain discretely adjusted portfolios.     

Stochastic invariance of closed sets with non-Lipschitz coefficients

This paper provides a new characterization of the stochastic invariance of a closed subset of ${\mathbb{R}}^d$ with respect to a diffusion. We extend the well-known inward pointing Stratonovich drift condition to the case where the diffusion matrix can fail to be differentiable: we only assume that the covariance matrix is. In particular, our result can be applied to construct affine and polynomial diffusions on any arbitrary closed set.     

Stochastic Approximation Algorithms for Parameter Estimation in Option Pricing with Regime Switching

Abstract

This work is concerned with option pricing. Stochastic approximation/optimization algorithms are proposed and analyzed. The underlying stock price evolves according to two geometric Brownian motions coupled by a continuous-time finite state Markov chain. Recursive stochastic approximation algorithms are developed to estimate the implied volatility. Convergence of the algorithm is proved. Rate of convergence is also ascertained. Then real market data are used to compare our algorithms with other schemes.     

Stochastic Taylor Expansions for Functionals of Diffusion Processes

In this article, a stochastic Taylor expansion of some functional applied to the solution process of an Itô or Stratonovich stochastic differential equation with a multi-dimensional driving Wiener process is given. Therefore, the multi-colored rooted tree analysis is applied in order to obtain a transparent representation of the expansion which is similar to the B-series expansion for solutions of ordinary differential equations in the deterministic setting. Further, some estimates for the mean-square and the mean truncation errors are given.