Bias and variance reduction techniques for bootstrap information criteria

We discuss the problem of constructing information criteria by applying the bootstrap methods. Various bias and variance reduction methods are presented for improving the bootstrap bias correction term in computing the bootstrap information criterion. The properties of these methods are investigated both in theoretical and numerical aspects, for which we use a statistical functional approach. It is shown that the bootstrap method automatically achieves the second-order bias correction if the bias of the first-order bias correction term is properly removed. We also show that the variance associated with bootstrapping can be considerably reduced for various model estimation procedures without any analytical argument. Monte Carlo experiments are conducted to investigate the performance of the bootstrap bias and variance reduction techniques.     

An efficient shrinkage bootstrap bias estimator for smooth functions of sample means

Summary  Since it is not always possible to calculate bootstrap estimators, they are usually approximated by simulation. In this article, we propose a bootstrap bias estimator for smooth functions of sample means that has less mean squared error, due to the simulation process, than the ordinary bootstrap. The estimator is based on shrinking the bootstrap mean towards the original sample mean. It can easily be implemented while demanding almost no additional computational effort.     

Bias correction for estimated distortion risk measure using the bootstrap

The bias of the empirical estimate of a given risk measure has recently been of interest in the risk management literature. In particular, Kim and Hardy (2007) showed that the bias can be corrected for the Conditional Tail Expectation (CTE, a.k.a. Tail-VaR or Expected Shortfall) using the bootstrap. This article extends their result to the distortion risk measure (DRM) class where the CTE is a special case. In particular, through the exact bootstrap, it is analytically proved that the bias of the empirical estimate of DRM with concave distortion function is negative and can be corrected on the bootstrap, using the fact that the bootstrapped loss is majorized by the original loss vector. Since the class of DRM is a subset of the L-estimator class, the result provides a sufficient condition for the bootstrap bias correction for L-estimators. Numerical examples are presented to show the effectiveness of the bootstrap bias correction. Later a practical guideline to choose the estimate with a lower mean squared error is also proposed based on the analytic form of the double bootstrapped estimate, which can be useful in estimating risk measures where the bias is non-cumulative across loss portfolio.     

Bootstrap for estimating the MSE of the Spatial EBLUP

This work assumes that the small area quantities of interest follow a Fay–Herriot model with spatially correlated random area effects. Under this model, parametric and nonparametric bootstrap procedures are proposed for estimating the mean squared error of the empirical best linear unbiased predictor (EBLUP). A simulation study based on the Italian Agriculture Census 2000 compares bootstrap and analytical estimates of the MSE and studies their robustness to non-normality. Results indicate lower bias for the non-parametric bootstrap under specific departures from normality.      

Cross-validated bagged learning

Many applications aim to learn a high dimensional parameter of a data generating distribution based on a sample of independent and identically distributed observations. For example, the goal might be to estimate the conditional mean of an outcome given a list of input variables. In this prediction context, bootstrap aggregating (bagging) has been introduced as a method to reduce the variance of a given estimator at little cost to bias. Bagging involves applying an estimator to multiple bootstrap samples and averaging the result across bootstrap samples. In order to address the curse of dimensionality, a common practice has been to apply bagging to estimators which themselves use cross-validation, thereby using cross-validation within a bootstrap sample to select fine-tuning parameters trading off bias and variance of the bootstrap sample-specific candidate estimators. In this article we point out that in order to achieve the correct bias variance trade-off for the parameter of interest, one should apply the cross-validation selector externally to candidate bagged estimators indexed by these fine-tuning parameters. We use three simulations to compare the new cross-validated bagging method with bagging of cross-validated estimators and bagging of non-cross-validated estimators.     

On a mapping approach to investigating the bootstrap accuracy

Summary. A simple mapping approach is proposed to study the bootstrap accuracy in a rather general setting. It is demonstrated that the bootstrap accuracy can be obtained through this method for a broad class of statistics to which the commonly used Edgeworth expansion approach may not be successfully applied. We then consider some examples to illustrate how this approach may be used to find the bootstrap accuracy and show the advantage of the bootstrap approximation over the Gaussian approximation. For the multivariate Kolmogorov–Smirnov statistic, we show the error of bootstrap approximation is as small as that of the Gaussian approximation. For the multivariate kernel type density estimate, we obtain an order of the bootstrap error which is smaller than the order of the error of the Gaussian approximation given in Rio (1994). We also consider an application of the bootstrap accuracy for empirical process to that for the copula process. Received: 23 June 1995 / In revised form: 18 June 1996     

Bootstrap of the offspring mean in the critical process with a non-stationary immigration

In applications of branching processes, usually it is hard to obtain samples of a large size. Therefore, a bootstrap procedure allowing inference based on a small sample size is very useful. Unfortunately, in the critical branching process with stationary immigration the standard parametric bootstrap is invalid. In this paper, we consider a process with non-stationary immigration, whose mean and variance vary regularly with nonnegative exponents α and β, respectively. We prove that 1+2α is the threshold for the validity of the bootstrap in this model. If β<1+2α, the standard bootstrap is valid and if β>1+2α it is invalid. In the case β=1+2α, the validity of the bootstrap depends on the slowly varying parts of the immigration mean and variance. These results allow us to develop statistical inferences about the parameters of the process in its early stages.     

The exact bootstrap method shown on the example of the mean and variance estimation

The bootstrap method is based on resampling of the original randomsample drawn from a population with an unknown distribution. In the article it was shown that because of the progress in computer technology resampling is actually unnecessary if the sample size is not too large. It is possible to automatically generate all possible resamples and calculate all realizations of the required statistic. The obtained distribution can be used in point or interval estimation of population parameters or in testing hypotheses. We should stress that in the exact bootstrap method the entire space of resamples is used and therefore there is no additional bias which results from resampling. The method was used to estimate mean and variance. The comparison of the obtained distributions with the limit distributions confirmed the accuracy of the exact bootstrap method. In order to compare the exact bootstrap method with the basic method (with random sampling) probability that 1,000 resamples would allow for estimating a parameter with a given accuracy was calculated. There is little chance of obtaining the desired accuracy, which is an argument supporting the use of the exact method. Random sampling may be interpreted as discretization of a continuous variable.     

An Estimate of the Odds Ratio That Always Exists

This article proposes an estimate of the odds ratio in a (2 × 2) table obtained from studies in which the row totals are fixed by design, such as a phase II clinical trial. Our estimate, based on the median unbiased estimate of the probabilities of success in the (2× 2) table, will always be in the interval (0, ∞). Another estimate of the odds ratio which has such properties is obtained when adding .5 to each cell of the table. Using simulations, we compared our proposed estimate to that obtained by adding .5 to every cell, and found that our estimate had smaller finite sample bias, and larger mean square error. We also propose the use of the bootstrap to form a confidence interval for the odds ratio based on our proposed estimate. Instead of a Monte Carlo bootstrap, one can easily calculate the “exact” bootstrap distribution of our estimate of the odds ratio, and use this distribution to calculate confidence intervals.     

Bootstrap estimation of the asymptotic variances of statistical functionals

A modified bootstrap estimator of the asymptotic variance of a statistical functional is studied. The modified bootstrap variance estimator circumvents the problem of the original bootstrap when the population distribution has heavy tails, and requires less stringent conditions for its consistency than the ordinary bootstrap variance estimator. The consistency of the modified bootstrap variance estimator is established for differentiable statistical functionals.     

Bootstrapped efficiency estimates for a model for groups and hierarchies in DEA

We apply some recently introduced bootstrap techniques to derive bias corrected efficiency scores for a model for groups and hierarchies in DEA. The use of the bootstrap makes it possible to overcome some deficiencies of the original formulation of this model, which rests on rescaling individual efficiency scores using average efficiencies calculated from different subsets of the data. These average or structural efficiencies are differently biased and bias varies with sample size when standard DEA techniques are used. Bias correction makes it possible to identify the true differences in efficiency and thus to compare DMUs belonging to different groups via their rescaled individual efficiency scores on one common basis. Moreover, this type of bias problem is present in other DEA applications. Therefore, the method proposed to deal with it has many potential applications beyond the groups and hierarchies model.     

交替迭代最大似然估计量的偏倚校正

In this paper, we give a definition of the alternating iterative maximum likelihood estimator (AIMLE) which is a biased estimator. Furthermore we adjust the AIMLE to result in asymptotically unbiased and consistent estimators by using a bootstrap iterative bias correction method as in Kuk (1995). Two examples and simulation results reported illustrate the performance of the bias correction for AIMLE.     

Studying the bandwidth in k-sample smooth tests

In this paper, the problem of bandwidth choice in smooth k-sample tests is considered. Three different bootstrap methods are discussed and implemented. All the methods persecute the bandwidth leading to the maximum power, while preserving the level of the test. The relative performance of the methods is investigated in a simulation study. Illustration through real medical data is provided. The main conclusion is that the bootstrap minimum method provides a good compromise between statistical power and conservativeness. Robustness of the methods with respect to the number of bootstrap resamples and practical limitations are discussed.     

Resampling DEA estimates of investment fund performance

Data envelopment analysis (DEA) is attractive for comparing investment funds because it handles different characteristics of fund distribution and gives a way to rank funds. There is substantial literature applying DEA to funds, based on the time series of funds’ returns. This article looks at the issue of uncertainty in the resulting DEA efficiency estimates, investigating consistency and bias. It uses the bootstrap to develop stochastic DEA models for funds, derive confidence intervals and develop techniques to compare and rank funds and represent the ranking. It investigates how to deal with autocorrelation in the time series and considers models that deal with correlation in the funds’ returns.     

Comparison on five estimation approaches of intensity for a queueing system with short run

Traffic intensity is an important measure for assessing performance of a queueing system. In this paper, we propose a consistent and asymptotically normal estimator (CAN) of intensity for a queueing system with distribution-free interarrival and service times. Using this estimator and its estimated variance, a 100(1 ? α)% asymptotic confidence interval of the intensity is constructed. Also, four bootstrap approaches—standard bootstrap, Bayesian bootstrap, percentile bootstrap, and bias-corrected and accelerated bootstrap are also applied to develop the confidence intervals of the intensity. A comparative analysis is conducted to demonstrate performances of the five confidence intervals of the intensity for a queueing system with short run data.     

The bootstrap: Some large sample theory and connections with robustness

The bootstrap, discussed by Efron (1979, 1981), is a powerful tool for the nonparametric estimation of sampling distributions and asymptotic standard errors. We demonstrate consistency of the bootstrap distribution estimates for a general class of robust differentiable statistical functionals. Our conditions for consistency of the bootstrap are variants of previously considered criteria for robustness of the associated statistics. A general example shows that, for almost any location statistic, consistency of the bootstrap variance estimator requires a tail condition on the distribution from which samples are taken. A modification of Efron's estimator of standard error is shown to circumvent this problem.     

The validity of least squares estimation in a time series model using the bootstrap methodology

The boostrap methodology may be used for estimating standard errors of the estimated parameters in a time series model. The idea is to approximate the theoretical error distribution by the residual distribution. The main objective of this article is to demonstrate the use of the bootstrap to attach standard errors to coefficient estimates in a second-order auto-regressive model fitted by least squares estimation. A comparison of the conventional and bootstrap methodology is made. A numerical result shows that the traditional least squares asymptotic formula for estimating standard errors appear to overestimate the true standard errors. But there are two problems in the simulation world of bootstrap for the autoregressive model of order two: (1) the first two observations y₁ and y₂ have been fixed, and (2) the residuals have not been inflated. After these two factors are considered in the trial and bootstrap experiment, both the conventional least squares and bootstrap estimates of the standard errors appear to be performing quite well.     

A test for a parametric form of the volatility in second-order diffusion models

Second-order diffusion models have been found to be promising in analyzing financial market data. Based on nonparametric fitting, Nicolau (Stat Probabil Lett 78(16):2700–2704, 2008) suggested that the quadratic function may be an appropriate specification of the volatility when a second-order diffusion model is used to analyze some European and American financial market data sets, which motivates us to develop a formal statistical test for this finding. To achieve the task, a generalized likelihood ratio test is proposed in this paper and a residual-based bootstrap is suggested to compute the p value of the test. The analysis of many real-world financial market data sets demonstrates that the quadratic specification of the volatility function is in general reasonable.     

Statistical inference for DEA estimators of directional distances

In productivity and efficiency analysis, the technical efficiency of a production unit is measured through its distance to the efficient frontier of the production set. The most familiar non-parametric methods use Farrell–Debreu, Shephard, or hyperbolic radial measures. These approaches require that inputs and outputs be non-negative, which can be problematic when using financial data. Recently, Chambers et al. (1998) have introduced directional distance functions which can be viewed as additive (rather than multiplicative) measures efficiency. Directional distance functions are not restricted to non-negative input and output quantities; in addition, the traditional input and output-oriented measures are nested as special cases of directional distance functions. Consequently, directional distances provide greater flexibility. However, until now, only free disposal hull (FDH) estimators of directional distances (and their conditional and robust extensions) have known statistical properties (Simar and Vanhems, 2012). This paper develops the statistical properties of directional d estimators, which are especially useful when the production set is assumed convex. We first establish that the directional Data Envelopment Analysis (DEA) estimators share the known properties of the traditional radial DEA estimators. We then use these properties to develop consistent bootstrap procedures for statistical inference about directional distance, estimation of confidence intervals, and bias correction. The methods are illustrated in some empirical examples.     

BOOTSTRAP WAVELET IN THE NONPARAMETRIC REGRESSION MODEL WITH WEAKLY DEPENDENT PROCESSES

This paper introduces a method of bootstrap wavelet estimation in a nonparametric regression model with weakly dependent processes for both fixed and random designs. The asymptotic bounds for the bias and variance of the bootstrap wavelet estimators are given in the fixed design model. The conditional normality for a modified version of the bootstrap wavelet estimators is obtained in the fixed model. The consistency for the bootstrap wavelet estimator is also proved in the random design model. These results show that the bootstrap wavelet method is valid for the model with weakly dependent processes.