The exact bootstrap method shown on the example of the mean and variance estimation

The bootstrap method is based on resampling of the original randomsample drawn from a population with an unknown distribution. In the article it was shown that because of the progress in computer technology resampling is actually unnecessary if the sample size is not too large. It is possible to automatically generate all possible resamples and calculate all realizations of the required statistic. The obtained distribution can be used in point or interval estimation of population parameters or in testing hypotheses. We should stress that in the exact bootstrap method the entire space of resamples is used and therefore there is no additional bias which results from resampling. The method was used to estimate mean and variance. The comparison of the obtained distributions with the limit distributions confirmed the accuracy of the exact bootstrap method. In order to compare the exact bootstrap method with the basic method (with random sampling) probability that 1,000 resamples would allow for estimating a parameter with a given accuracy was calculated. There is little chance of obtaining the desired accuracy, which is an argument supporting the use of the exact method. Random sampling may be interpreted as discretization of a continuous variable.     

Resampling time series using missing values techniques

Several techniques for resampling dependent data have already been proposed. In this paper we use missing values techniques to modify the moving blocks jackknife and bootstrap. More specifically, we consider the blocks of deleted observations in the blockwise jackknife as missing data which are recovered by missing values estimates incorporating the observation dependence structure. Thus, we estimate the variance of a statistic as a weighted sample variance of the statistic evaluated in a “complete” series. Consistency of the variance and the distribution estimators of the sample mean are established. Also, we apply the missing values approach to the blockwise bootstrap by including some missing observations among two consecutive blocks and we demonstrate the consistency of the variance and the distribution estimators of the sample mean. Finally, we present the results of an extensive Monte Carlo study to evaluate the performance of these methods for finite sample sizes, showing that our proposal provides variance estimates for several time series statistics with smaller mean squared error than previous procedures.     

On the estimation of a monotone conditional variance in nonparametric regression

A monotone estimate of the conditional variance function in a heteroscedastic, nonparametric regression model is proposed. The method is based on the application of a kernel density estimate to an unconstrained estimate of the variance function and yields an estimate of the inverse variance function. The final monotone estimate of the variance function is obtained by an inversion of this function. The method is applicable to a broad class of nonparametric estimates of the conditional variance and particularly attractive to users of conventional kernel methods, because it does not require constrained optimization techniques. The approach is also illustrated by means of a simulation study.     

Estimating Precision in Functional Images

Abstract

Functional imaging of biologic parameters like in vivo tissue metabolism is made possible by Positron Emission Tomography (PET). Many techniques have been suggested for extracting such images from dynamic time-course sequences of reconstructed PET scans. Quantitating the precision of these estimates is important for drawing inferences on the biologic parameters. Analytic variance formulas are not immediate owing to the nonlinear methods used in extraction. The usual resampling approach is infeasible because each image reconstruction in PET is a computationally demanding solution to a high-dimensional linear inverse problem. We suggest an alternative simulation approach that approximates the distribution of reconstructed PET scans and performs a parametric bootstrap in the imaging domain. Results on a simplified model chosen to match the characteristics of PET reconstruction are very encouraging. Mixture analysis is used to estimate functional images; however, the suggested approach is general enough to extend to other techniques or imaging methods.     

Premium rates based on genetic studies: How reliable are they?

Underwriting the risk of rare disorders in long-term insurance often relies on rates of onset estimated from quite small epidemiological studies. These estimates can have considerable sampling uncertainty and any function based upon them, such as a premium rate, is also an estimate subject to uncertainty. This is particularly relevant in the case of genetic disorders, because the acceptable use of genetic information may depend on establishing its reliability as a measure of risk. The sampling distribution of a premium rate is hard to estimate without access to the original data, which is rarely possible. From two studies of adult polycystic kidney disease (APKD) we obtain, not the original data, but the cases and exposures used for Kaplan-Meier estimates of the survival probability. We use three resampling methods with these data, namely: (a) the standard bootstrap; (b) the weird bootstrap; and (c) simulation of censored random lifetimes. Rates of onset were obtained from each simulated sample using kernel-smoothed Nelson-Aalen estimates, hence critical illness insurance premium rates for a mutation carrier or a member of an affected family. From 10,000 such samples we estimate the sampling distributions of the premium rates, finding considerable uncertainty. Very careful consideration should be given before using small-sample epidemiological data to deal with insurance problems.     

Sampling per mode for rare event simulation in switching diffusions

A straightforward application of an interacting particle system to estimate a rare event for switching diffusions fails to produce reasonable estimates within a reasonable amount of simulation time. To overcome this, a conditional “sampling per mode” algorithm has been proposed by Krystul in [10]; instead of starting the algorithm with particles randomly distributed, we draw in each mode, a fixed number particles and at each resampling step, the same number of particles is sampled for each visited mode. In this paper, we establish a law of large numbers as well as a central limit theorem for the estimate.     

Real-Time Density and Mode Estimation With Application to Time-Dynamic Mode Tracking

We introduce a nonparametric time-dynamic kernel type density estimate for the situation where an underlying multivariate distribution evolves with time. Based on this time-dynamic density estimate, we propose nonparametric estimates for the time-dynamic mode of the underlying distribution. Our estimators involve boundary kernels for the time dimension so that the estimator is always centered at current time, and multivariate kernels for the spatial dimension of the time-evolving distribution. Under certain mild conditions, the asymptotic behavior of density and mode estimators, especially their uniform convergence in both time and space, is derived. A time-dynamic algorithm for mode tracking is proposed, including automatic bandwidth choices, and is implemented via a mean update algorithm. Simulation studies and real data illustrations demonstrate that the proposed methods work well in practice.     

Applications of the representative points in statistical simulations

The paper gives a new approach to statistical simulation and resampling by the use of number-theoretic methods and representative points. Resempling techniques take samples from an approximate population. The bootstrap suggests to use a random sample to form an approximate population. We propose to construct some approximate population distribution by the use of two kinds of representative points, and samples are taken from these approximate distributions. The statistical inference is based on those samples. The statistical inference in this paper involves estimation of mean, variance, skewness, kurtosis, quantile and density of the population distribution. Our results show that the new method can significantly improve the results by the use of Monte Carlo methods.     

Mixtures of Global and Local Edgeworth Expansions and Their Applications

Edgeworth expansions which are local in one coordinate and global in the rest of the coordinates are obtained for sums of independent but not identically distributed random vectors. Expansions for conditional probabilities are deduced from these. Both lattice and continuous conditioning variables are considered. The results are then applied to derive Edgeworth expansions for bootstrap distributions, for Bayesian bootstrap distribution, and for the distributions of statistics based on samples from finite populations. This results in a unified theory of Edgeworth expansions for resampling procedures. The Bayesian bootstrap is shown to be second order correct for smooth positive “priors,” whenever the third cumulant of the “prior” is equal to the third power of its standard deviation. Similar results are established for weighted bootstrap when the weights are constructed from random variables with a lattice distribution.     

Estimation of the radius of a circle when the coordinates of a number of points on its circumference are observed: An example of bootstrapping

The bootstrap (Efron, 1979, 1982) is a very simple resampling plan and has shown to be successful in estimating the bias and other measures of statistical error of a number of estimators. It gives freedom from the constraints of traditional parametric theory at the cost of performing the usual statistical calculations a hundred or a thousand times over. In this letter another example of bootstrap estimation of bias is given. It is interesting that Quenouille's (1949) jackknife (see Miller, 1964, for a review) fails completely in this case.     

The Bumpy Road to the Mode Forest

Abstract

The mode tree of Minnotte and Scott provides a valuable method of investigating features such as modes and bumps in a unknown density. By examining kernel density estimates for a range of bandwidths, we can learn a lot about the structure of a data set. Unfortunately, the basic mode tree can be strongly affected by small changes in the data, and gives no way to differentiate between important modes and those caused, for example, by outliers. The mode forest overcomes these difficulties by looking simultaneously at a large collection of mode trees, all based on some variation of the original data, by means such as resampling or jittering. The resulting graphic tool is both visually appealing and informative.     

The Mode Tree: A Tool for Visualization of Nonparametric Density Features

Abstract

Recognition and extraction of features in a nonparametric density estimate are highly dependent on correct calibration. The data-driven choice of bandwidth h in kernel density estimation is a difficult one that is compounded by the fact that the globally optimal h is not generally optimal for all values of x. In recognition of this fact a new type of graphical tool, the mode tree, is proposed. The basic mode tree plot relates the locations of modes in density estimates with the bandwidths of those estimates. Additional information can be included on the plot indicating factors such as the size of modes, how modes split, and the locations of antimodes and bumps. The use of a mode tree in adaptive multimodality investigations is proposed, and an example is given to show the value in using a normal kernel, as opposed to the biweight or other kernels, in such investigations. Examples of such investigations are provided for Ahrens's chondrite data and van Winkle's Hidalgo stamp data. Finally, the bivariate mode tree is introduced, together with an example using Scott's lipid data.     

Hybrid bootstrap for mapping quantitative trait loci

The hybrid bootstrap uses resampling ideas to extend the duality approach to interval estimation for a parameter of interest when there are nuisance parameters. The confidence region constructed by the hybrid bootstrap may perform much better than the parametric bootstrap region in situations where the data provide substantial information about the nuisance parameter, but limited information about the parameter of interest. We apply this method to estimate the location of quantitative trait loci (QTL) in interval mapping model. The conditional distribution of quantitative traits, given flanked genetic marker genotypes is often assumed to be the mixture model of two phenotype distributions. The mixing proportions in the model represent the recombination rate between a genetic marker and quantitative trait loci and provides information about the unknown location of the QTL. Since recombination events are unlikely, we will have less information about the location of the QTL than other parameters. This observation makes a hybrid approach to interval estimation for QTL appealing, especially since the necessary distribution theory, which is often a challenge for mixture models, can be handled by bootstrap simulation.     

Outlier detection in data envelopment analysis: an analysis of jackknifing

This paper analyzes the resampling technique of jackknifing and its capability of detecting outliers in data envelopment analysis. It is well recognized that measured efficiency is sensitive to outliers; recent research has employed resampling techniques to estimate standard deviations in an attempt to handle outliers. Using jackknifing, each observation other than the decision making unit under analysis is deleted from the sample once and the resulting linear program is solved, leading to a distribution of efficiency estimates. From this distribution, standard deviations and confidence intervals are derived. Two types of outliers can be distinguished conceptually: those belonging to the production possibility set that are efficient, and those that do not belong but appear to due to statistical noise. This paper argues that calculation of the standard deviation is not meaningful because it is not possible to distinguish empirically between the two types of outliers.     

Bayesian nonparametric predictive inference and bootstrap techniques

We address the question as to whether a prior distribution on the space of distribution functions exists which generates the posterior produced by Efron's and Rubin's bootstrap techniques, emphasizing the connection with the Dirichlet process. We also introduce a new resampling plan which has two advantages: prior opinions are taken into account and the predictive distribution of the future observations is not forced to be concentrated on observed values.     

Dimension Asymptotics for Generalised Bootstrap in Linear Regression

We prove consistency of a class of generalised bootstrap techniques for the distribution of the least squares parameter estimator in linear regression, when the number of parameters tend to infinity with data size and the regressors are random. We show that best results are obtainable with resampling techniques that have not been considered earlier in the literature.     

Spectral approximation for compact integral operators by degenerate kernel methods

We establish a new improved error estimate for the solution of the integral equation eigenvalue problem by degenerate kernel methods. In [6] these estimates were proved under the assumption of normality of the original kernel as well as of the approximating degenerate kernel. Now we consider any compact integral operator and a general Banach space situation, in contrast to the Hilbert space setting in [6], This will be done by combining the techniques in [6] with the suitably transformed estimates of [5]. Our results show that degenerate kernel methods have, besides their overall property of furnishing easy approximations to eigenfunctions, for eigenvalues an order of convergence comparable to quadrature methods.     

Density Estimation under Constraints

Abstract

We suggest a general method for tackling problems of density estimation under constraints. It is, in effect, a particular form of the weighted bootstrap, in which resampling weights are chosen so as to minimize distance from the empirical or uniform bootstrap distribution subject to the constraints being satisfied. A number of constraints are treated as examples. They include conditions on moments, quantiles, and entropy, the latter as a device for imposing qualitative conditions such as those of unimodality or “interestingness.” For example, without altering the data or the amount of smoothing, we may construct a density estimator that enjoys the same mean, median, and quartiles as the data. Different measures of distance·give rise to slightly different results.     

Nonparametric Resampling for Homogeneous Strong Mixing Random Fields

Künsch (1989, Ann. Statist.17 1217-1241) and Liu ane Singh (1992, in Exploring Limits of Bootstrap (R. Le Page and L. Billard, Eds.), pp. 225-248, Wiley, New York) have recently introduced a block resampling method that is successful in deriving consistent bootstrap estimates of distribution and variance for the sample mean of a strong mixing sequence. Raïs and Moore (1990, in Interface ′90) and Raïs (1992, Ph.D. Thesis, University of Montreal) extended the results of Künsch and Liu and Singh in the case of the sample mean of a homogeneous strong mixing random field in two dimensions (n = 2). In this paper, the general case (n Z⁺) is considered, and a resampling technique for strong mixing random fields is formulated, which is an extension of the "blocks of blocks" resampling scheme for sequences in Politis and Romano (1992, Ann. Statist.20 (4) 1985-2007). The "blocks of blocks" method can be used to construct asymptotically correct confidence intervals for parameters of the whole (infinite-dimensional) joint distribution of the random field, for example, the spectral density at a point. A variation of the "blocks of blocks" resampling scheme that involves "wrapping" the data around on a torus will also be studied, in view of its property to yield an unbiased bootstrap distribution.     

Bootstrap bandwidth selection in kernel density estimation from a contaminated sample

In this paper we consider kernel estimation of a density when the data are contaminated by random noise. More specifically we deal with the problem of how to choose the bandwidth parameter in practice. A theoretical optimal bandwidth is defined as the minimizer of the mean integrated squared error. We propose a bootstrap procedure to estimate this optimal bandwidth, and show its consistency. These results remain valid for the case of no measurement error, and hence also summarize part of the theory of bootstrap bandwidth selection in ordinary kernel density estimation. The finite sample performance of the proposed bootstrap selection procedure is demonstrated with a simulation study. An application to a real data example illustrates the use of the method. This research was supported by ‘Projet d’Actions de Recherche Concertées’ (No. 98/03-217) from the Belgian government. Financial support from the IAP research network nr P5/24 of the Belgian State (Federal Office for Scientific, Technical and Cultural Affairs) is also gratefully acknowledged.