Speeding up MCMC by Delayed Acceptance and Data Subsampling

The complexity of the Metropolis–Hastings (MH) algorithm arises from the requirement of a likelihood evaluation for the full dataset in each iteration. One solution has been proposed to speed up the algorithm by a delayed acceptance approach where the acceptance decision proceeds in two stages. In the first stage, an estimate of the likelihood based on a random subsample determines if it is likely that the draw will be accepted and, if so, the second stage uses the full data likelihood to decide upon final acceptance. Evaluating the full data likelihood is thus avoided for draws that are unlikely to be accepted. We propose a more precise likelihood estimator that incorporates auxiliary information about the full data likelihood while only operating on a sparse set of the data. We prove that the resulting delayed acceptance MH is more efficient. The caveat of this approach is that the full dataset needs to be evaluated in the second stage. We therefore propose to substitute this evaluation by an estimate and construct a state-dependent approximation thereof to use in the first stage. This results in an algorithm that (i) can use a smaller subsample m by leveraging on recent advances in Pseudo-Marginal MH (PMMH) and (ii) is provably within O(m^{? 2}) of the true posterior.     

两水平无重复因析试验散度效应的估计

本文对两水平无重复因析试验给出了散度效应的一种新的估计,称为AMH估计,改进了文献中散度效应的较好的MH估计的一个缺陷,给出并证明了AMH估计的无偏条件,证明了AMH估计比MH估计有更小的方差下界.最后通过模拟试验比较了AMH和MH估计的偏度,方差和均方误差.     

无重复因析试验中一种散度效应的迭代估计方法

在无重复因析试验下,基于李济洪,王钰等(2010)提出的散度效应的AMH估计,给出了一种散度效应的迭代估计方法(称为IAMH估计),通过模拟试验验证了此方法比常用的AMH,MH估计具有更小的均方误差.     

Adaptive,Delayed-Acceptance MCMC for Targets With Expensive Likelihoods

When conducting Bayesian inference, delayed-acceptance (DA) Metropolis–Hastings (MH) algorithms and DA pseudo-marginal MH algorithms can be applied when it is computationally expensive to calculate the true posterior or an unbiased estimate thereof, but a computationally cheap approximation is available. A first accept-reject stage is applied, with the cheap approximation substituted for the true posterior in the MH acceptance ratio. Only for those proposals that pass through the first stage is the computationally expensive true posterior (or unbiased estimate thereof) evaluated, with a second accept-reject stage ensuring that detailed balance is satisfied with respect to the intended true posterior. In some scenarios, there is no obvious computationally cheap approximation. A weighted average of previous evaluations of the computationally expensive posterior provides a generic approximation to the posterior. If only the k-nearest neighbors have nonzero weights then evaluation of the approximate posterior can be made computationally cheap provided that the points at which the posterior has been evaluated are stored in a multi-dimensional binary tree, known as a KD-tree. The contents of the KD-tree are potentially updated after every computationally intensive evaluation. The resulting adaptive, delayed-acceptance [pseudo-marginal] Metropolis–Hastings algorithm is justified both theoretically and empirically. Guidance on tuning parameters is provided and the methodology is applied to a discretely observed Markov jump process characterizing predator–prey interactions and an ODE system describing the dynamics of an autoregulatory gene network. Supplementary material for this article is available online.     

随机删失下概率密度核估计的光滑Bootstrap逼近

本文研究随机删失概率密度估计的光ｂｏｏｔｓｔｒａｐ逼近。给出了光滑ｂｏｏｔｓｔｒａｐ逼近成立的充分条件，并证明了概率密度的光滑ｂｏｏｔｓｔｒａｐ估计方差几乎处处收敛到概率密度核估计的渐近方差。     

An Adaptive Version for the Metropolis Adjusted Langevin Algorithm with a Truncated Drift

This paper extends some adaptive schemes that have been developed for the Random Walk Metropolis algorithm to more general versions of the Metropolis-Hastings (MH) algorithm, particularly to the Metropolis Adjusted Langevin algorithm of Roberts and Tweedie (1996). Our simulations show that the adaptation drastically improves the performance of such MH algorithms. We study the convergence of the algorithm. Our proves are based on a new approach to the analysis of stochastic approximation algorithms based on mixingales theory.      

无重复因析试验中一种估计散度效应的方法

自从Box和Meyer首次提出无重复因析试验中散度效应的识别和估计问题, 各种散度效应的估计方法(包括迭代和非迭代)被提出. 特别地, Brenneman 和Nair 给出了这些方法的一个综述, 并且他们验证了改进的Harvey方法优于其它的方法.本文中对于对数线性模型, 一个基于多个位置模型残差平均的非迭代的散度效应估计方法在模型选择阶段被提出. 在大多数的模拟实验模型中, 本文方法具有比MH方法更小的均方误差, 且它可以应用于MH方法不适用的0或小的绝对残差情形. 我们也考虑了这个估计的理论性质, 并进行了实例分析.     

The Distribution of Robust Distances

Mahalanobis-type distances in which the shape matrix is derived from a consistent, high-breakdown robust multivariate location and scale estimator have an asymptotic chi-squared distribution as is the case with those derived from the ordinary covariance matrix. For example, Rousseeuw's minimum covariance determinant (MCD) is a robust estimator with a high breakdown. However, even in quite large samples, the chi-squared approximation to the distances of the sample data from the MCD center with respect to the MCD shape is poor. We provide an improved F approximation that gives accurate outlier rejection points for various sample sizes.     

Adaptive FEM and a posteriori error control for stationary coupled bulk-surface PDEs

We consider a system of two coupled elliptic equations, one defined on a bulk domain and the other one on the boundary surface. The numerical error of the finite element solution can be controlled by a residual a posteriori error estimator which takes into account the approximation errors due to the discretisation in space as well as the polyhedral approximation of the surface. The estimators naturally lead to refinement indicators for an adaptive algorithm to control the overall error. Numerical experiments illustrate the performance of the a posteriori error estimator and the adaptive algorithm. (© 2016 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

A general ratio estimator and its application in model based inference

A general ratio estimator of a population total is proposed as an approximation to the estimator introduced by Srivastava (1985,Bull. Internat. Statist. Inst.,51(10.3), 1–16). This estimator incorporates additional information gathered during the survey in a new way. Statistical properties of the general ratio estimator are given and its relationship to the estimator proposed by Srivastava is explored. A special kind of general ratio estimator is suggested and it turns out to be very efficient in a simulation study when compared to several other commonly used estimators.The work of this author was supported by AFOSR grant #830080.     

Statistical Inference in Single-Index and Partially Nonlinear Models

A finite series approximation technique is introduced. We first applythis approximation technique to a semiparametric single-index model toconstruct a nonlinear least squares (LS) estimator for an unknown parameterand then discuss the confidence region for this parameter based on theasymptotic distribution of the nonlinear LS estimator. Meanwhile, acomputational algorithm and a small sample study for this nonlinear LSestimator are developed. Additionally, we apply the finite seriesapproximation technique to a partially nonlinear model and obtain some newresults.     

Efficient computation of generalized median estimators

The generalized median (GM) estimator is a family of robust estimators that balances the competing demands of statistical efficiency and robustness. By choosing a kernel that is efficient for the parameter, the GM estimator gains robustness by computing the median of the kernel evaluated at all possible subsets from the sample. The GM estimator is often computationally infeasible because the number of subsets can be large for even modest sample sizes. Writing the estimator in terms of the quantile function facilitates an approximation using a sample of all possible subsets. While both sampling with and without replacement are feasible, sampling without replacement is preferred because of the reduction in variance from the sampling fraction. The proposed algorithm uses sequential sampling to compute an approximation within a user-chosen margin of error.     

Asymptotic analysis for a stochastic semidefinite programming

Stochastic semidefinite programming (SSDP) is a new class of optimization problems with a wide variety of applications. In this article, asymptotic analysis results of sample average approximation estimator for SSDP are established. Asymptotic analysis result already existing for stochastic nonlinear programming is extended to SSDP, that is, the conditions ensuring the convergence in distribution of sample average approximation estimator for SSDP to a multivariate normal are obtained and the corresponding covariance matrix is described in a closed form.     

Bias Reduction for Pricing American Options by Least-Squares Monte Carlo

Abstract

We derive an approximation to the bias in regression-based Monte Carlo estimators of American option values. This derivation holds for general asset-price processes of any dimensionality and for general pay-off structures. It uses the large sample properties of least-squares regression estimators. Bias-corrected estimators result by subtracting the bias approximation from the uncorrected estimator at each exercise opportunity. Numerical results show that the bias-corrected estimator outperforms its uncorrected counterpart across all combinations of number of exercise opportunities, option moneyness and sample size. Finally, the results suggest significant computational efficiency increases can be realized through trivial parallel implementations using the corrected estimator.     

Asymptotically optimal choice of the smoothing parameter in a nonparametric kernel estimator for a probability density

This paper presents a method of estimation of an “optimal” smoothing parameter (window width) in kernel estimators for a probability density. The obtained estimator is calculated directly from observations. By “optimal” smoothing parameters we mean those parameters which minimize the mean integral square error (MISE) or the integral square error (ISE) of approximation of an unknown density by the kernel estimator. It is shown that the asymptotic “optimality” properties of the proposed estimator correspond (with respect to the order) to those of the well-known cross-validation procedure [1, 2]. Translated fromStatisticheskie Metody Otsenivaniya i Proverki Gipotez, pp. 67–80, Perm, 1990.     

两水平无重复因析试验散度效应BH估计的性质

本文研究了两水平无重复因析试验散度效应BH估计的性质,给出了BH估计无偏性的充分必要条件,求得了它的近似方差.并在多个模型下对BH与MH估计进行了模拟比较.     

Edgeworth expansion for the survival function estimator in the Koziol-Green model

In the Koziol-Green or proportional hazards random censorship model, the asymptotic accuracy of the estimated one-term Edgeworth expansion and the smoothed bootstrap approximation for the Studentized Abdushukurov-Cheng-Lin estimator is investigated. It is shown that both the Edgeworth expansion estimate and the bootstrap approximation are asymptotically closer to the exact distribution of the Studentized Abdushukurov-Cheng-Lin estimator than the normal approximation.     

The Reduced Basis Method for an Elastic Buckling Problem

In this work, we apply the Reduced Basis (RB) Method to the field of nonlinear elasticity. In this first stage of research, we analyze a buckling problem for a compressed 2D column: Here, the trivial linear solution is computed for an arbitrary load; the critical load, marking the transition to nonlinearity, is then identified through an eigenvalue problem. The linear problem satisfies the Lax-Milgram conditions, allowing the implementation of both a Successive Constraint Method for an inexpensive lower bound of the coercivity constant and of a rigorous and efficient a posteriori error estimator for the RB approximation. Even though only a non-rigorous estimator is available for the buckling problem, the actual RB approximation of the output is more than satisfactory, and the gain in computational efficiency significant. (© 2013 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

APPROXIMATION RATES OF ERROR DISTRIBUTION OF DOUBLE KERNEL ESTIMATES OF CONDITIONAL DENSITY

§ 1　IntroductionLet(X,Y) be a random vector taking values Rp×Rqand assume that with given X=x,f(y|x) is the conditional density of Y,the Borel-measurable function on(x,y) ,X has amarginal distribution function F(x) and a marginal density function f(x) .Let(X1 ,Y1 ) ,...,(Xn,Yn) be i.i.d.sample taking values in(X,Y) .A class of double kernel esti-mates of f(y|x) proposed by Zhao Linchang and Liu Zhijun[1 ] has the formfn(y|x) = ni=1K1Xi -xan K2Yi -ybn bqn nj=1K1Xj-xan ,(1 .1 )where…     

条件密度双重核估计误差分布的逼近速度

In this paper, the normal approximation rate and the random weighting approximation rate of error distribution of the kernel estimator of conditional density function f(y!|x) are studied. The results may be used to construct the confidence interval of f(y|x).