BOOTSTRAP MAXIMUM LIKELIHOOD ESTIMATION OF THE PARAMETER IN SPECTRAL DENSITY OF STATIONARY PROCESSES

ＢＯＯＴＳＴＲＡＰ　ＭＡＸＩＭＵＭＬＩＫＥＬＩＨＯＯＤＥＳＴＩＭＡＴＩＯＮＯＦＴＨＥＰＡＲＡＭＥＴＥＲＩＮＳＰＥＣＴＲＡＬＤＥＮＳＩＴＹＯＦＳＴＡＴＩＯＮＡＲＹ　ＰＲＯＣＥＳＳＥＳＹＵＤＡＮ（于丹）（ＩｎｓｔｉｔｕｔｅｏｆＳｙｓｔｅｍｓＳｃｉｅｎｃｅ...     

Confidence Intervals and Accuracy Estimation for Heavy-Tailed Generalized Pareto Distributions

The generalized Pareto distribution (GPD) is a two-parameter family of distributions which can be used to model exceedances over a threshold. We compare the empirical coverage of some standard bootstrap and likelihood-based confidence intervals for the parameters and upper p-quantiles of the GPD. Simulation results indicate that none of the bootstrap methods give satisfactory intervals for small sample sizes. By applying a general method of D. N. Lawley, correction factors for likelihood ratio statistics of parameters and quantiles of the GPD have been calculated. Simulations show that for small sample sizes accuracy of confidence intervals can be improved by incorporating the computed correction factors to the likelihood-based confidence intervals. While the modified likelihood method has better empirical coverage probability, the mean length of produced intervals are not longer than corresponding bootstrap confidence intervals. This article also investigates the performance of some bootstrap methods for estimation of accuracy measures of maximum likelihood estimators of parameters and quantiles of the GPD.     

Cross-validated bagged learning

Many applications aim to learn a high dimensional parameter of a data generating distribution based on a sample of independent and identically distributed observations. For example, the goal might be to estimate the conditional mean of an outcome given a list of input variables. In this prediction context, bootstrap aggregating (bagging) has been introduced as a method to reduce the variance of a given estimator at little cost to bias. Bagging involves applying an estimator to multiple bootstrap samples and averaging the result across bootstrap samples. In order to address the curse of dimensionality, a common practice has been to apply bagging to estimators which themselves use cross-validation, thereby using cross-validation within a bootstrap sample to select fine-tuning parameters trading off bias and variance of the bootstrap sample-specific candidate estimators. In this article we point out that in order to achieve the correct bias variance trade-off for the parameter of interest, one should apply the cross-validation selector externally to candidate bagged estimators indexed by these fine-tuning parameters. We use three simulations to compare the new cross-validated bagging method with bagging of cross-validated estimators and bagging of non-cross-validated estimators.     

Efficient construction of a smooth nonparametric family of empirical distributions and calculation of bootstrap likelihood

This paper considers the efficient construction of a nonparametric family of distributions indexed by a specified parameter of interest and its application to calculating a bootstrap likelihood for the parameter. An approximate expression is obtained for the variance of log bootstrap likelihood for statistics which are defined by an estimating equation resulting from the method of selecting the first-level bootstrap populations and parameters. The expression is shown to agree well with simulations for artificial data sets based on quantiles of the standard normal distribution, and these results give guidelines for the amount of aggregation of bootstrap samples with similar parameter values required to achieve a given reduction in variance. An application to earthquake data illustrates how the variance expression can be used to construct an efficient Monte Carlo algorithm for defining a smooth nonparametric family of empirical distributions to calculate a bootstrap likelihood by greatly reducing the inherent variability due to first-level resampling.     

Hybrid bootstrap for mapping quantitative trait loci

The hybrid bootstrap uses resampling ideas to extend the duality approach to interval estimation for a parameter of interest when there are nuisance parameters. The confidence region constructed by the hybrid bootstrap may perform much better than the parametric bootstrap region in situations where the data provide substantial information about the nuisance parameter, but limited information about the parameter of interest. We apply this method to estimate the location of quantitative trait loci (QTL) in interval mapping model. The conditional distribution of quantitative traits, given flanked genetic marker genotypes is often assumed to be the mixture model of two phenotype distributions. The mixing proportions in the model represent the recombination rate between a genetic marker and quantitative trait loci and provides information about the unknown location of the QTL. Since recombination events are unlikely, we will have less information about the location of the QTL than other parameters. This observation makes a hybrid approach to interval estimation for QTL appealing, especially since the necessary distribution theory, which is often a challenge for mixture models, can be handled by bootstrap simulation.     

Bootstrap for the conditional distribution function with truncated and censored data

We propose a resampling method for left truncated and right censored data with covariables to obtain a bootstrap version of the conditional distribution function estimator. We derive an almost sure representation for this bootstrapped estimator and, as a consequence, the consistency of the bootstrap is obtained. This bootstrap approximation represents an alternative to the normal asymptotic distribution and avoids the estimation of the complicated mean and variance parameters of the latter.     

Estimating critical values for testing the i.i.d. in standardized residuals from GARCH models in finite samples

Taking into account that the BDS test—which is used as a misspecification test applied to standardized residuals from the GARCH(1,1) model—is characterized by size distortion and departure from normality in finite samples, this paper obtains the critical values for the finite sample distribution of the BDS test. We focus on bootstrap simulation to avoid the sampling uncertainty of parameter estimation and make use of estimated response surface regressions (RSR) derived from the experimental results. We consider an extensive grid of models to obtain critical values with the results of the bootstrap experiments. The RSR used to estimate them is an artificial neural network (ANN) model, instead of the traditional linear regression models. Specifically, we estimate critical values by using a bootstrap aggregated neural network (BANN) and by employing functions of the sample size and parameters used in the experiment as the embedding dimension and proximity parameters in the BDS statistic, GARCH parameters and even the q-quantiles of the BDS distributions. The main results confirm that the sample size and BDS parameters play a role in size distortion. Finally, an empirical application to three price indexes is performed, to highlight the differences between decisions made using the asymptotic or our predicted critical values for the BDS test in finite samples.     

A Bootstrap Approach to Nonparametric Regression for Right Censored Data

In this paper a two-stage bootstrap method is proposed for nonparametric regression with right censored data. The method is applied to construct confidence intervals and bands for a conditional survival function. Its asymptotic validity is established using counting process techniques and martingale central limit theory. The performance of the bootstrap method is investigated in a Monte Carlo study. An illustration is given using a real data.     

The exact bootstrap method shown on the example of the mean and variance estimation

The bootstrap method is based on resampling of the original randomsample drawn from a population with an unknown distribution. In the article it was shown that because of the progress in computer technology resampling is actually unnecessary if the sample size is not too large. It is possible to automatically generate all possible resamples and calculate all realizations of the required statistic. The obtained distribution can be used in point or interval estimation of population parameters or in testing hypotheses. We should stress that in the exact bootstrap method the entire space of resamples is used and therefore there is no additional bias which results from resampling. The method was used to estimate mean and variance. The comparison of the obtained distributions with the limit distributions confirmed the accuracy of the exact bootstrap method. In order to compare the exact bootstrap method with the basic method (with random sampling) probability that 1,000 resamples would allow for estimating a parameter with a given accuracy was calculated. There is little chance of obtaining the desired accuracy, which is an argument supporting the use of the exact method. Random sampling may be interpreted as discretization of a continuous variable.     

On resampling techniques for regression models

Resampling techniques like the bootstrap are examined for functions of the parameters of a linear model. A weighted resampling method analogous to the weighted jackknife developed by Hinkley (1977) is proposed for regression models.     

The bootstrap and Bayesian bootstrap method in assessing bioequivalence

Parametric method for assessing individual bioequivalence (IBE) may concentrate on the hypothesis that the PK responses are normal. Nonparametric method for evaluating IBE would be bootstrap method. In 2001, the United States Food and Drug Administration (FDA) proposed a draft guidance. The purpose of this article is to evaluate the IBE between test drug and reference drug by bootstrap and Bayesian bootstrap method. We study the power of bootstrap test procedures and the parametric test procedures in FDA (2001). We find that the Bayesian bootstrap method is the most excellent.     

Exact inference for joint Type-I hybrid censoring model with exponential competing risks data

Assuming that the failure time under different risk factors follows the independent exponential distribution, a joint model under Type-I hybrid censoring is addressed in detail. Based on the Maximum likelihood estimates (MLEs) of unknown parameters, we obtain exact distributions of MLEs by using the moment generating function (MGF). Confidence intervals (CIs) of parameters are constructed through both the exact method and the parametric bootstrap method. Then we compare the performances of different methods by Monte Carlo simulations. Finally, the validity of the proposed models and methods are demonstrated by a numerical example.     

Improving solve time of aggregation‐based adaptive AMG

This paper proposes improving the solve time of a bootstrap algebraic multigrid (AMG) designed previously by the authors. This is achieved by incorporating the information, a set of algebraically smooth vectors, generated by the bootstrap algorithm, in a single hierarchy by using sufficiently large aggregates, and these aggregates are compositions of aggregates already built throughout the bootstrap algorithm. The modified AMG method has good convergence properties and shows significant reduction in both memory and solve time. These savings with respect to the original bootstrap AMG are illustrated on some difficult (for standard AMG) linear systems arising from discretization of scalar and vector function elliptic partial differential equations in both 2D and 3D.     

A new resampling method for sampling designs without replacement: the doubled half bootstrap

A new and very fast method of bootstrap for sampling without replacement from a finite population is proposed. This method can be used to estimate the variance in sampling with unequal inclusion probabilities and does not require artificial populations or utilization of bootstrap weights. The bootstrap samples are directly selected from the original sample. The bootstrap procedure contains two steps: in the first step, units are selected once with Poisson sampling using the same inclusion probabilities as the original design. In the second step, amongst the non-selected units, half of the units are randomly selected twice. This procedure enables us to efficiently estimate the variance. A set of simulations show the advantages of this new resampling method.     

A sequential method for the development of visual interactive meta-simulation models using neural networks

This paper proposes a practical and efficient method for the development of visual interactive meta-simulation models using neural networks. The method first uses a randomised simulation experimental design to obtain a set of results from a previously validated simulation model. The bootstrap technique is used on these results to generate a series of neural network models that are then trained using back propagation. The visual interactive meta-simulation model consists of the collective response from the trained neural network models. The accuracy of the meta-simulation model is assessed using the bootstrap technique and improved accuracy obtained by increasing the size of the randomised simulation experimental design set and re-training. This paper describes the approach, gives results for five example problems and suggests that the method is a practical extension to visual interactive simulation.     

Inference for a progressive stress model from Weibull distribution under progressive type-II censoring

Based on progressively type-II censored samples, this paper considers progressive stress accelerated life tests when the lifetime of an item under use condition follows the Weibull distribution with a scale parameter satisfying the inverse power law. It is assumed that the progressive stress is directly proportional to time and the cumulative exposure model for the effect of changing stress holds. Point estimation of the model parameters is obtained graphically by using Weibull probability paper plot that serves as a tool for model identification and also by using the maximum likelihood method. Interval estimation is performed by finding approximate confidence intervals (CIs) for the parameters as well as the studentized-t and percentile bootstrap CIs. Monte Carlo simulation study is carried out to investigate the precision of the estimates and compare the performance of CIs obtained. Finally, two examples are presented to illustrate our results.     

Second-order accuracy of depth-based bootstrap confidence regions

We consider the problem of setting bootstrap confidence regions for multivariate parameters based on data depth functions. We prove, under mild regularity conditions, that depth-based bootstrap confidence regions are second-order accurate in the sense that their coverage error is of order n⁻¹, given a random sample of size n. The results hold in general for depth functions of types A and D, which cover as special cases the Tukey depth, the majority depth, and the simplicial depth. A simulation study is also provided to investigate empirically the bootstrap confidence regions constructed using these three depth functions.     

Some Methods for Estimation in a Negative-Binomial Model

To clarify the advantage of using the quasilikelihood method, lack of robustness of the maximum likelihood method was demonstrated for the negative-binomial model. Efficiency calculations of the method of moments and the pseudolikelihood method in the estimation of extra-Poisson parameters in a negative-binomial model were carried out. Especially when the overdispersion parameter is small, both methods are relatively highly efficient and the pseudolikelihood estimate is more efficient than the method of moments estimate. Two examples of the quasilikelihood analyses of count data with overdispersion are given. The bootstrap method also is applied to the data to illustrate the advantage of the method of moments or pseudolikelihood method in the estimation of the standard errors of the mean parameter estimates under the negative-binomial model.     

Statistical Analysis in Constant-Stress Accelerated Life Tests for Generalized Exponential Distribution Based on Adaptive Type-II Progressive Hybrid Censored Data

Based on adaptive type-II progressive hybrid censored data statistical analysis for constant-stress accelerated life test (CS-ALT) with products' lifetime following two-parameter generalized exponential (GE) distribution is investigated. The estimates of the unknown parameters and the reliability function are obtained through a new method combining the EM algorithm and the least square method. The observed Fisher information matrix is achieved with missing information principle, and the asymptotic unbiased estimate (AUE) of the scale parameter is also obtained. Confidence intervals (CIs) for the parameters are derived using asymptotic normality of the estimators and the percentile bootstrap (Boot-p) method. Finally, Monte Carlo simulation study is carried out to investigate the precision of the point estimates and interval estimates, respectively. It is shown that the AUE of the scale parameter is better than the corresponding two-step estimation, and the Boot-p CIs are more accurate than the corresponding asymptotic CIs.     

Random weighting method for Cox’s proportional hazards model

Variance of parameter estimate in Cox’s proportional hazards model is based on asymptotic variance. When sample size is small, variance can be estimated by bootstrap method. However, if censoring rate in a survival data set is high, bootstrap method may fail to work properly. This is because bootstrap samples may be even more heavily censored due to repeated sampling of the censored observations. This paper proposes a random weighting method for variance estimation and confidence interval estimation for proportional hazards model. This method, unlike the bootstrap method, does not lead to more severe censoring than the original sample does. Its large sample properties are studied and the consistency and asymptotic normality are proved under mild conditions. Simulation studies show that the random weighting method is not as sensitive to heavy censoring as bootstrap method is and can produce good variance estimates or confidence intervals.