Lower bounds for bandwidth selection in density estimation

Summary This paper establishes asymptotic lower bounds which specify, in a variety of contexts, how well (in terms of relative rate of convergence) one may select the bandwidth of a kernel density estimator. These results provide important new insights concerning how the bandwidth selection problem should be considered. In particular it is shown that if the error criterion is Integrated Squared Error (ISE) then, even under very strong assumptions on the underlying density, relative error of the selected bandwidth cannot be reduced below ordern ^–1/10 (as the sample size grows). This very large error indicates that any technique which aims specifically to minimize ISE will be subject to serious practical difficulties arising from sampling fluctuations. Cross-validation exhibits this very slow convergence rate, and does suffer from unacceptably large sampling variation. On the other hand, if the error criterion is Mean Integrated Squared Error (MISE) then relative error of bandwidth selection can be reduced to ordern ^–1/2, when enough smoothness is assumed. Therefore bandwidth selection techniques which aim to minimize MISE can be much more stable, and less sensitive to small sampling fluctuations, than those which try to minimize ISE. We feel this indicates that performance in minimizing MISE, rather than ISE, should become the benchmark for measuring performance of bandwidth selection methods.Research partially supported by National Science Foundation Grants DMS-8701201 and DMS-8902973Research of the first author was done while on leave from the Australian National University     

变系数联立模型的局部线性广义矩变窗宽估计

在随机设计条件下,提出了一类变系数联立模型,运用局部线性广义矩变窗宽估计,对模型的变系数进行了估计,研究了估计量的大样本性质.利用概率论中大数定律和中心极限定理,证明了估计量的大样本性质,局部线性广义矩变窗宽估计具有相合性和渐进正态性.     

Bootstrap bandwidth selection in kernel density estimation from a contaminated sample

In this paper we consider kernel estimation of a density when the data are contaminated by random noise. More specifically we deal with the problem of how to choose the bandwidth parameter in practice. A theoretical optimal bandwidth is defined as the minimizer of the mean integrated squared error. We propose a bootstrap procedure to estimate this optimal bandwidth, and show its consistency. These results remain valid for the case of no measurement error, and hence also summarize part of the theory of bootstrap bandwidth selection in ordinary kernel density estimation. The finite sample performance of the proposed bootstrap selection procedure is demonstrated with a simulation study. An application to a real data example illustrates the use of the method. This research was supported by ‘Projet d’Actions de Recherche Concertées’ (No. 98/03-217) from the Belgian government. Financial support from the IAP research network nr P5/24 of the Belgian State (Federal Office for Scientific, Technical and Cultural Affairs) is also gratefully acknowledged.     

Kernel density estimation for linear processes: Asymptotic normality and optimal bandwidth derivation

The problem of estimating the marginal density of a linear process by kernel methods is considered. Under general conditions, kernel density estimators are shown to be asymptotically normal. Their limiting covariance matrix is computed. We also find the optimal bandwidth in the sense that it asymptotically minimizes the mean square error of the estimators. The assumptions involved are easily verifiable.Supported in part by NSF grant DMS-9403718.     

Local Walsh-average-based estimation and variable selection for single-index models

We propose a robust estimation procedure based on local Walsh-average regression(LWR) for single-index models. Our novel method provides a root-n consistent estimate of the single-index parameter under some mild regularity conditions; the estimate of the unknown link function converges at the usual rate for the nonparametric estimation of a univariate covariate. We theoretically demonstrate that the new estimators show significant efficiency gain across a wide spectrum of non-normal error distributions and have almost no loss of efficiency for the normal error. Even in the worst case, the asymptotic relative efficiency(ARE) has a lower bound compared with the least squares(LS) estimates; the lower bounds of the AREs are 0.864 and 0.8896 for the single-index parameter and nonparametric function, respectively. Moreover, the ARE of the proposed LWR-based approach versus the ARE of the LS-based method has an expression that is closely related to the ARE of the signed-rank Wilcoxon test as compared with the t-test. In addition, to obtain a sparse estimate of the single-index parameter, we develop a variable selection procedure by combining the estimation method with smoothly clipped absolute deviation penalty; this procedure is shown to possess the oracle property. We also propose a Bayes information criterion(BIC)-type criterion for selecting the tuning parameter and further prove its ability to consistently identify the true model. We conduct some Monte Carlo simulations and a real data analysis to illustrate the finite sample performance of the proposed methods.     

On estimation and inference in a partially linear hazard model with varying coefficients

We study estimation and inference in a marginal proportional hazards model that can handle (1) linear effects, (2) non-linear effects and (3) interactions between covariates. The model under consideration is an amalgamation of three existing marginal proportional hazards models studied in the literature. Developing an estimation and inference procedure with desirable properties for the amalgamated model is rather challenging due to the co-existence of all three effects listed above. Much of the existing literature has avoided the problem by considering narrow versions of the model. The object of this paper is to show that an estimation and inference procedure that accommodates all three effects is within reach. We present a profile partial-likelihood approach for estimating the unknowns in the amalgamated model with the resultant estimators of the unknown parameters being root- \(n\) consistent and the estimated functions achieving optimal convergence rates. Asymptotic normality is also established for the estimators.     

Local linear kernel estimation of the discontinuous regression function

We address the problems of estimating the discontinuous regression function and also its jump points. We propose a method in two steps: we first estimate the jumps and finally the regression function is estimated by an adapted version of a local linear smoother which makes use of the estimated jumps. The practical performance of the proposed method is evaluated by using simulation studies and an application to a real-life problem.     

Some contributions to selection and estimation in the normal linear model

We study the choice of the quantity in the FPE criterion for selecting a member of a class of normal linear models having an orthogonal structure. Two approaches are discussed, namely fixing the maximal estimation risk at a prescribed level and using minimax regret. Estimation of the risk actually achieved and an illustrative example are also discussed.This research was supported by the FRD of South Africa.     

Bayesian bandwidth estimation for a semi-functional partial linear regression model with unknown error density

In the context of semi-functional partial linear regression model, we study the problem of error density estimation. The unknown error density is approximated by a mixture of Gaussian densities with means being the individual residuals, and variance a constant parameter. This mixture error density has a form of a kernel density estimator of residuals, where the regression function, consisting of parametric and nonparametric components, is estimated by the ordinary least squares and functional Nadaraya–Watson estimators. The estimation accuracy of the ordinary least squares and functional Nadaraya–Watson estimators jointly depends on the same bandwidth parameter. A Bayesian approach is proposed to simultaneously estimate the bandwidths in the kernel-form error density and in the regression function. Under the kernel-form error density, we derive a kernel likelihood and posterior for the bandwidth parameters. For estimating the regression function and error density, a series of simulation studies show that the Bayesian approach yields better accuracy than the benchmark functional cross validation. Illustrated by a spectroscopy data set, we found that the Bayesian approach gives better point forecast accuracy of the regression function than the functional cross validation, and it is capable of producing prediction intervals nonparametrically.     

Density and hazard rate estimation for censored data via strong representation of the Kaplan-Meier estimator

Summary We study the estimation of a density and a hazard rate function based on censored data by the kernel smoothing method. Our technique is facilitated by a recent result of Lo and Singh (1986) which establishes a strong uniform approximation of the Kaplan-Meier estimator by an average of independent random variables. (Note that the approximation is carried out on the original probability space, which should be distinguished from the Hungarian embedding approach.) Pointwise strong consistency and a law of iterated logarithm are derived, as well as the mean squared error expression and asymptotic normality, which is obtain using a more traditional method, as compared with the Hajek projection employed by Tanner and Wong (1983).     

Multivariate hazard rate orders

Two multivariate hazard rate stochastic orders are introduced and studied. Their meaning, properties, and relationship to other common stochastic orders are examined and investigated. Some examples that illustrate the theory are detailed. Finally, some applications of the new orders in reliability theory and in actuarial science are described.     

On the hazard rate and reversed hazard rate orderings in two-component series systems with active redundancies

The lifetimes of two-component series systems with two active redundancies are compared using the hazard rate and the reversed hazard rate orders. We study the problem of where to allocate the spares in a system to obtain the best configuration. We compare redundancy at component level vs. system level using the likelihood ratio order. For this problem we find conditions under which there is no hazard rate ordering between the lifetimes of the systems.     

A vector multivariate hazard rate

A vector definition of multivariate hazard rate, and associated definitions of increasing and decreasing multivariate hazard rate distributions are presented. Consequences of these definitions are worked out in a number of special cases. Relationships between hazard rate and orthant dependence are established.     

Local influence analysis for penalized Gaussian likelihood estimation in partially linear single-index models

Single-index model is a potentially tool for multivariate nonparametric regression, generalizes both the generalized linear models(GLM) and the missing-link function problem in GLM. In this paper, we extend Cook’s local influence analysis to the penalized Gaussian likelihood estimator based on P-spline for the partially linear single-index model. Some influence measures, based on the minor perturbation of the model, are derived for the penalized least squares estimation. An illustrative example is also presented.     

Local asymptotic normality and estimation via Kalman-Bucy filter for linear systems driven by fractional Brownian motions

We study the local asymptotic normality and estimation for drift parameter obtained through Kalman–Bucy filter for linear systems driven by fractional Brownian motions.     

Kernel quantile estimator with ICI adaptive bandwidth selection technique

In this article, we consider a class of kernel quantile estimators which is the linear combi- nation of order statistics. This class of kernel quantile estimators can be regarded as an extension of some existing estimators. The exact mean square error expression for this class of estimators will be provided when data are uniformly distributed. The implementation of these estimators depends mostly on the bandwidth selection. We then develop an adaptive method for bandwidth selection based on the intersection confidence intervals （ICI） principle. Monte Carlo studies demonstrate that our proposed approach is comparatively remarkable. We illustrate our method with a real data set.     

Duality of linear estimation and control

The concept of duality in mathematical programming is utilized to define a generalized duality principle between linear, minimum-variance estimation and fixed-time, linear-quadratic regulator problems which provides a mathematical basis for the duality noted by Kalman as a special case. This generalized duality is then used with results in colored-noise filtering to specify the solution to a class of singular regulator problems.This research was supported by the Air Force Office of Scientific Research, Grant No. AF-AFOSR-699-67.