The bootstrap: Some large sample theory and connections with robustness

The bootstrap, discussed by Efron (1979, 1981), is a powerful tool for the nonparametric estimation of sampling distributions and asymptotic standard errors. We demonstrate consistency of the bootstrap distribution estimates for a general class of robust differentiable statistical functionals. Our conditions for consistency of the bootstrap are variants of previously considered criteria for robustness of the associated statistics. A general example shows that, for almost any location statistic, consistency of the bootstrap variance estimator requires a tail condition on the distribution from which samples are taken. A modification of Efron's estimator of standard error is shown to circumvent this problem.     

BOOTSTRAP WAVELET IN THE NONPARAMETRIC REGRESSION MODEL WITH WEAKLY DEPENDENT PROCESSES

This paper introduces a method of bootstrap wavelet estimation in a nonparametric regression model with weakly dependent processes for both fixed and random designs. The asymptotic bounds for the bias and variance of the bootstrap wavelet estimators are given in the fixed design model. The conditional normality for a modified version of the bootstrap wavelet estimators is obtained in the fixed model. The consistency for the bootstrap wavelet estimator is also proved in the random design model. These results show that the bootstrap wavelet method is valid for the model with weakly dependent processes.     

Bootstrap for the conditional distribution function with truncated and censored data

We propose a resampling method for left truncated and right censored data with covariables to obtain a bootstrap version of the conditional distribution function estimator. We derive an almost sure representation for this bootstrapped estimator and, as a consequence, the consistency of the bootstrap is obtained. This bootstrap approximation represents an alternative to the normal asymptotic distribution and avoids the estimation of the complicated mean and variance parameters of the latter.     

Quantile residual lifetime for left-truncated and right-censored data

This article proposes a simple nonparametric estimator of quantile residual lifetime function under left-truncated and right-censored data. The asymptotic consistency and normality of this estimator are proved and the variance expression is calculated. Two bootstrap procedures are employed in the simulation study, where the latter bootstrap from Zeng and Lin (2008) is 4000 times faster than the former naive one, and the numerical results in both methods show that our estimating approach works well. A real data example is used to illustrate its application.     

随机删失下概率密度核估计的光滑Bootstrap逼近

本文研究随机删失概率密度估计的光ｂｏｏｔｓｔｒａｐ逼近。给出了光滑ｂｏｏｔｓｔｒａｐ逼近成立的充分条件，并证明了概率密度的光滑ｂｏｏｔｓｔｒａｐ估计方差几乎处处收敛到概率密度核估计的渐近方差。     

A Wald-type variance estimation for the nonparametric distribution estimators for doubly censored data

We discuss the variance estimation for the nonparametric distribution estimator for doubly censored data. We first provide another view of Kuhn–Tucker’s conditions to construct the profile likelihood, and lead a Newton–Raphson algorithm as an optimization technique unlike the EM algorithm. The main proposal is an iteration-free Wald-type variance estimate based on the chain rule of differentiating conditions to construct the profile likelihood, which generalizes the variance formula in only right- or left-censored data. In this estimation procedure, we overcome some difficulties caused in directly applying Turnbull’s formula to large samples and avoid a load with computationally heavy iterations, such as solving the Fredholm equations, computing the profile likelihood ratio or using the bootstrap. Also, we establish the consistency of the formulated Wald-type variance estimator. In addition, simulation studies are performed to investigate the properties of the Wald-type variance estimates in finite samples in comparison with those from the profile likelihood ratio.     

Bootstrapping the Kaplan–Meier estimator on the whole line

This article is concerned with proving the consistency of Efron’s bootstrap for the Kaplan–Meier estimator on the whole support of a survival function. While previous works address the asymptotic Gaussianity of the Kaplan–Meier estimator without restricting time, we enable the construction of bootstrap-based time-simultaneous confidence bands for the whole survival function. Other practical applications include bootstrap-based confidence bands for the mean residual lifetime function or the Lorenz curve as well as confidence intervals for the Gini index. Theoretical results are complemented with a simulation study and a real data example which result in statistical recommendations.

     

Comparison on five estimation approaches of intensity for a queueing system with short run

Traffic intensity is an important measure for assessing performance of a queueing system. In this paper, we propose a consistent and asymptotically normal estimator (CAN) of intensity for a queueing system with distribution-free interarrival and service times. Using this estimator and its estimated variance, a 100(1 ? α)% asymptotic confidence interval of the intensity is constructed. Also, four bootstrap approaches—standard bootstrap, Bayesian bootstrap, percentile bootstrap, and bias-corrected and accelerated bootstrap are also applied to develop the confidence intervals of the intensity. A comparative analysis is conducted to demonstrate performances of the five confidence intervals of the intensity for a queueing system with short run data.     

Cross-validated bagged learning

Many applications aim to learn a high dimensional parameter of a data generating distribution based on a sample of independent and identically distributed observations. For example, the goal might be to estimate the conditional mean of an outcome given a list of input variables. In this prediction context, bootstrap aggregating (bagging) has been introduced as a method to reduce the variance of a given estimator at little cost to bias. Bagging involves applying an estimator to multiple bootstrap samples and averaging the result across bootstrap samples. In order to address the curse of dimensionality, a common practice has been to apply bagging to estimators which themselves use cross-validation, thereby using cross-validation within a bootstrap sample to select fine-tuning parameters trading off bias and variance of the bootstrap sample-specific candidate estimators. In this article we point out that in order to achieve the correct bias variance trade-off for the parameter of interest, one should apply the cross-validation selector externally to candidate bagged estimators indexed by these fine-tuning parameters. We use three simulations to compare the new cross-validated bagging method with bagging of cross-validated estimators and bagging of non-cross-validated estimators.     

ASYMPTOTIC BEHAVIOR OF BOOTSTRAP SPECTRAL WINDOW ESTIMATION

ThisprojectissupportedbytheNationalNaturalScienceFoundationofChina.1.IntroductionLetX~{X(t);t=0,FI,12,'}beazerthmeantunitvariancestationaryGaussianprocess,theautocovariancefunction7(u)=EX(n u)X(n)satisfySupposethatthespectraldensityfunctionisf(A)andspectraldistributionfunctionisF(A)oftheprocessX,wherewerestrictAwithin11(if=[--x,7]).Fromtheassumptionsabove,wecaneasilyseethatthef(A)isjustaprobabilitydensityfunction,andF(A)isaprobabilitydistributionfunction.LetX(1),'tX(n)betheobserv…     

An exact bootstrap for variance of finite-population <Emphasis Type="BoldItalic">L</Emphasis>-statistic

We give an exact formula of a finite-population bootstrap variance estimator for a general class of L-statistic. It is aimed to reduce the computational burden and to eliminate the approximation error, typically present in resampling approximations based on simulation. In the case of the classical nonparametric Efron bootstrap, a similar formula was shown by Hutson and Ernst [A.D. Hutson and M.D. Ernst, The exact bootstrap mean and variance of an L-estimator, J. R. Stat. Soc., Ser. B, 62:89–94, 2000].     

Exact convergence rate of bootstrap quantile variance estimator

Summary It is shown that the relative error of the bootstrap quantile variance estimator is of precise order n ^-1/4, when n denotes sample size. Likewise, the error of the bootstrap sparsity function estimator is of precise order n ^-1/4. Therefore as point estimators these estimators converge more slowly than the Bloch-Gastwirth estimator and kernel estimators, which typically have smaller error of order at most n ^-2/5.     

Variance estimation for sample quantiles using them out ofn bootstrap

We consider the problem of estimating the variance of a sample quantile calculated from a random sample of sizen. Ther-th-order kernel-smoothed bootstrap estimator is known to yield an impressively small relative error of orderO(n ^−r/(2r+1) ). It nevertheless requires strong smoothness conditions on the underlying density function, and has a performance very sensitive to the precise choice of the bandwidth. The unsmoothed bootstrap has a poorer relative error of orderO(n ^−1/4), but works for less smooth density functions. We investigate a modified form of the bootstrap, known as them out ofn bootstrap, and show that it yields a relative error of order smaller thanO(n ^−1/4) under the same smoothness conditions required by the conventional unsmoothed bootstrap on the density function, provided that the bootstrap sample sizem is of an appropriate order. The estimator permits exact, simulation-free, computation and has accuracy fairly insensitive to the precise choice ofm. A simulation study is reported to provide empirical comparison of the various methods. Supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. HKU 7131/00P).     

Bias and variance reduction techniques for bootstrap information criteria

We discuss the problem of constructing information criteria by applying the bootstrap methods. Various bias and variance reduction methods are presented for improving the bootstrap bias correction term in computing the bootstrap information criterion. The properties of these methods are investigated both in theoretical and numerical aspects, for which we use a statistical functional approach. It is shown that the bootstrap method automatically achieves the second-order bias correction if the bias of the first-order bias correction term is properly removed. We also show that the variance associated with bootstrapping can be considerably reduced for various model estimation procedures without any analytical argument. Monte Carlo experiments are conducted to investigate the performance of the bootstrap bias and variance reduction techniques.     

Random weighting method for Cox’s proportional hazards model

Variance of parameter estimate in Cox’s proportional hazards model is based on asymptotic variance. When sample size is small, variance can be estimated by bootstrap method. However, if censoring rate in a survival data set is high, bootstrap method may fail to work properly. This is because bootstrap samples may be even more heavily censored due to repeated sampling of the censored observations. This paper proposes a random weighting method for variance estimation and confidence interval estimation for proportional hazards model. This method, unlike the bootstrap method, does not lead to more severe censoring than the original sample does. Its large sample properties are studied and the consistency and asymptotic normality are proved under mild conditions. Simulation studies show that the random weighting method is not as sensitive to heavy censoring as bootstrap method is and can produce good variance estimates or confidence intervals.     

Dimension Asymptotics for Generalised Bootstrap in Linear Regression

We prove consistency of a class of generalised bootstrap techniques for the distribution of the least squares parameter estimator in linear regression, when the number of parameters tend to infinity with data size and the regressors are random. We show that best results are obtainable with resampling techniques that have not been considered earlier in the literature.     

Two tests for heteroscedasticity in nonparametric regression

In this paper, two new tests for heteroscedasticity in nonparametric regression are presented and compared. The first of these tests consists in first estimating nonparametrically the unknown conditional variance function and then using a classical least-squares test for a general linear model to test whether this function is a constant. The second test is based on using an overall distance between a nonparametric estimator of the conditional variance function and a parametric estimator of the variance of the model under the assumption of homoscedasticity. A bootstrap algorithm is used to approximate the distribution of this test statistic. Extended versions of both procedures in two directions, first, in the context of dependent data, and second, in the case of testing if the variance function is a polynomial of a certain degree, are also described. A broad simulation study is carried out to illustrate the finite sample performance of both tests when the observations are independent and when they are dependent.     

变系数EV模型基于核估计的误差方差估计

在利用核函数法和广义最小二乘法讨论变系数EV模型系数参数估计的基础上给出了其误差方差σ2的一种估计量(σ)2n,证明了所定义的估计量(σ)2n有很好的大样本性质.     

Estimation and Bootstrap with Censored Data in Fixed Design Nonparametric Regression

We study Beran's extension of the Kaplan-Meier estimator for thesituation of right censored observations at fixed covariate values. Thisestimator for the conditional distribution function at a given value of thecovariate involves smoothing with Gasser-Müller weights. We establishan almost sure asymptotic representation which provides a key tool forobtaining central limit results. To avoid complicated estimation ofasymptotic bias and variance parameters, we propose a resampling methodwhich takes the covariate information into account. An asymptoticrepresentation for the bootstrapped estimator is proved and the strongconsistency of the bootstrap approximation to the conditional distributionfunction is obtained.     

Statistical inference of partially linear regression models with heteroscedastic errors

The authors study a heteroscedastic partially linear regression model and develop an inferential procedure for it. This includes a test of heteroscedasticity, a two-step estimator of the heteroscedastic variance function, semiparametric generalized least-squares estimators of the parametric and nonparametric components of the model, and a bootstrap goodness of fit test to see whether the nonparametric component can be parametrized.