FFT-Based Fast Computation of Multivariate Kernel Density Estimators With Unconstrained Bandwidth Matrices

The problem of fast computation of multivariate kernel density estimation (KDE) is still an open research problem. In our view, the existing solutions do not resolve this matter in a satisfactory way. One of the most elegant and efficient approach uses the fast Fourier transform. Unfortunately, the existing FFT-based solution suffers from a serious limitation, as it can accurately operate only with the constrained (i.e., diagonal) multivariate bandwidth matrices. In this article, we describe the problem and give a satisfactory solution. The proposed solution may be successfully used also in other research problems, for example, for the fast computation of the optimal bandwidth for KDE. Supplementary materials for this article are available online.     

Error Process Indexed by Bandwidth Matrices in Multivariate Local Linear Smoothing

We focus on nonparametric multivariate regression function estimation by locally weighted least squares. The asymptotic behavior for a sequence of error processes indexed by bandwidth matrices is derived. We discuss feasible data-driven consistent estimators minimizing asymptotic mean squared error or efficient estimators reducing asymptotic bias at points where opposite sign curvatures of the regression function are present in different directions.     

Plug-in method for nonparametric regression

The problem of bandwidth selection for non-parametric kernel regression is considered. We will follow the Nadaraya–Watson and local linear estimator especially. The circular design is assumed in this work to avoid the difficulties caused by boundary effects. Most of bandwidth selectors are based on the residual sum of squares (RSS). It is often observed in simulation studies that these selectors are biased toward undersmoothing. This leads to consideration of a procedure which stabilizes the RSS by modifying the periodogram of the observations. As a result of this procedure, we obtain an estimation of unknown parameters of average mean square error function (AMSE). This process is known as a plug-in method. Simulation studies suggest that the plug-in method could have preferable properties to the classical one. Supported by the MSMT: LC 06024.     

Assessing log-concavity of multivariate densities

We develop a test for log-concavity of multivariate densities. The method uses kernel density estimation, where the test statistic is the smallest bandwidth for which the estimate is log-concave. We examine the properties of this technique through numerical studies.     

Estimation of Wishart mean matrices under simple tree ordering

For Wishart density functions, we study the risk dominance problems of the restricted maximum likelihood estimators of mean matrices with respect to the Kullback-Leibler loss function over restricted parameter space under the simple tree ordering set. The results are directly applied to the estimation of covariance matrices for the completely balanced multivariate multi-way random effects models without interactions.     

基于局部多项式展开的多元非参数模型贝叶斯带宽选择

在多元非参数模型中带宽和阶的选择对局部多项式估计量的表现十分重要。本文基于交叉验证准则提出一个自适应贝叶斯带宽选择方法。在给定的误差密度函数下,该方法可推导出对应的似然函数,并构造带宽参数的后验密度函数。随后,通过带宽的后验期望可同时获得阶和带宽的估计。数值模拟的结果表明,该方法不仅比大拇指准则方法精确,且比交叉验证方法耗时更少。与此同时,与Nadaraya-Watson估计相比,所提带宽选择方法对多元非参数模型的适应性要更好。最后,本文通过一组实际数据说明有限样本下所提贝叶斯带宽选择的表现很好。     

Plug-in bandwidth selector for the kernel relative density estimator

This paper is focused on two kernel relative density estimators in a two-sample problem. An asymptotic expression for the mean integrated squared error of these estimators is found and, based on it, two solve- the-equation plug-in bandwidth selectors are proposed. In order to examine their practical performance a simulation study and a practical application to a medical dataset are carried out.     

A note on the universal consistency of the kernel distribution function estimator

The problem of universal consistency of data driven bandwidth selectors for the kernel distribution estimator is analyzed. We provide a uniform in bandwidth result for the kernel estimate of a continuous distribution function. Our smoothness assumption is minimal in the sense that if the true distribution function has some discontinuity then the kernel estimate is no longer consistent.     

Exact and Asymptotically Optimal Bandwidths for Kernel Estimation of Density Functionals

Given a density f we pose the problem of estimating the density functional $\psi_r=\int f^{(r)}f$ for a non-negative even r making use of kernel methods. This is a well-known problem but some of its features remained unexplored. We focus on the problem of bandwidth selection. Whereas all the previous studies concentrate on an asymptotically optimal bandwidth here we study the properties of exact, non-asymptotic ones, and relate them with the former. Our main conclusion is that, despite being asymptotically equivalent, for realistic sample sizes much is lost by using the asymptotically optimal bandwidth. In contrast, as a target for data-driven selectors we propose another bandwidth which retains the small sample performance of the exact one.     

Strong convergence in nonparametric regression with truncated dependent data

In this paper we derive rates of uniform strong convergence for the kernel estimator of the regression function in a left-truncation model. It is assumed that the lifetime observations with multivariate covariates form a stationary α-mixing sequence. The estimation of the covariate’s density is considered as well. Under the assumption that the lifetime observations are bounded, we show that, by an appropriate choice of the bandwidth, both estimators of the covariate’s density and regression function attain the optimal strong convergence rate known from independent complete samples.     

On the quadratic estimation of covariance matrices in multivariate linear models

This paper investigates the estimation of covariance matrices in multivariate mixed models. Some sufficient conditions are derived for a multivariate quadratic form and a linear combination of multivariate quadratic forms to be the BQUE (quadratic unbiased and severally minimum varianced) estimators of its expectations.     

Graphical Criticism: Some Historical Notes

This article proposes a bootstrap local bandwidth selector for estimating nonparametric additive models. The selector is derived from a bootstrap approximation of the conditional mean squared error, based on a wild bootstrap resampling scheme applied to the estimated residuals. The selector is computed exactly (without involving Monte Carlo approximations) and in practice can be evaluated for many additive estimation methods, including backfitting (bivariate), marginal integration, and mixed methods. We study the consistency of the bootstrap approximation and also carry out an empirical simulation study to explore the performance of the proposed selector in comparison with others. The graphical tool SiZer Map enables us to make meaningful comparisons between local and global selectors.     

Non-asymptotic Bandwidth Selection for Density Estimation of Discrete Data

We propose a new method for density estimation of categorical data. The method implements a non-asymptotic data-driven bandwidth selection rule and provides model sparsity not present in the standard kernel density estimation method. Numerical experiments with a well-known ten-dimensional binary medical data set illustrate the effectiveness of the proposed approach for density estimation, discriminant analysis and classification. Supported by the Australian Research Council, under grant number DP0558957.     

Maximum likelihood estimation of Wishart mean matrices under Löwner order restrictions

For Wishart density functions, there remains a long-time question unsolved. That is whether there exists the closed-form MLEs of mean matrices over the partially Löwner ordering sets. In this note, we provide an affirmative answer by demonstrating a unified procedure on exactly how the closed-form MLEs are obtained for the simple ordering case. Under the Kullback-Leibler loss function, a property of obtained MLEs is further studied. Some applications of the obtained closed-form MLEs, including the comparison between our ML estimates and Calvin and Dykstra's [Maximum likelihood estimation of a set of covariance matrices under Löwner order restrictions with applications to balanced multivariate variance components models, Ann. Statist. 19 (1991) 850-869.] which obtained by iterative algorithm, are also made.     

Locally adaptive hazard smoothing

Summary We consider nonparametric estimation of hazard functions and their derivatives under random censorship, based on kernel smoothing of the Nelson (1972) estimator. One critically important ingredient for smoothing methods is the choice of an appropriate bandwidth. Since local variance of these estimates depends on the point where the hazard function is estimated and the bandwidth determines the trade-off between local variance and local bias, data-based local bandwidth choice is proposed. A general principle for obtaining asymptotically efficient data-based local bandwiths, is obtained by means of weak convergence of a local bandwidth process to a Gaussian limit process. Several specific asymptotically efficient bandwidth estimators are discussed. We propose in particular an, asymptotically efficient method derived from direct pilot estimators of the hazard function and of the local mean squared error. This bandwidth choice method has practical advantages and is also of interest in the uncensored case as well as for density estimation.Research supported by UC Davis Faculty Research Grant and by Air Force grant AFOSR-89-0386Research supported by Air Force grant AFOSR-89-0386     

Bayesian bandwidth estimation for a semi-functional partial linear regression model with unknown error density

In the context of semi-functional partial linear regression model, we study the problem of error density estimation. The unknown error density is approximated by a mixture of Gaussian densities with means being the individual residuals, and variance a constant parameter. This mixture error density has a form of a kernel density estimator of residuals, where the regression function, consisting of parametric and nonparametric components, is estimated by the ordinary least squares and functional Nadaraya–Watson estimators. The estimation accuracy of the ordinary least squares and functional Nadaraya–Watson estimators jointly depends on the same bandwidth parameter. A Bayesian approach is proposed to simultaneously estimate the bandwidths in the kernel-form error density and in the regression function. Under the kernel-form error density, we derive a kernel likelihood and posterior for the bandwidth parameters. For estimating the regression function and error density, a series of simulation studies show that the Bayesian approach yields better accuracy than the benchmark functional cross validation. Illustrated by a spectroscopy data set, we found that the Bayesian approach gives better point forecast accuracy of the regression function than the functional cross validation, and it is capable of producing prediction intervals nonparametrically.     

Boundary kernels for adaptive density estimators on regions with irregular boundaries

In some applications of kernel density estimation the data may have a highly non-uniform distribution and be confined to a compact region. Standard fixed bandwidth density estimates can struggle to cope with the spatially variable smoothing requirements, and will be subject to excessive bias at the boundary of the region. While adaptive kernel estimators can address the first of these issues, the study of boundary kernel methods has been restricted to the fixed bandwidth context. We propose a new linear boundary kernel which reduces the asymptotic order of the bias of an adaptive density estimator at the boundary, and is simple to implement even on an irregular boundary. The properties of this adaptive boundary kernel are examined theoretically. In particular, we demonstrate that the asymptotic performance of the density estimator is maintained when the adaptive bandwidth is defined in terms of a pilot estimate rather than the true underlying density. We examine the performance for finite sample sizes numerically through analysis of simulated and real data sets.     

Challenging the curse of dimensionality in multivariate local linear regression

Local polynomial fitting for univariate data has been widely studied and discussed, but up until now the multivariate equivalent has often been deemed impractical, due to the so-called curse of dimensionality. Here, rather than discounting it completely, we use density as a threshold to determine where over a data range reliable multivariate smoothing is possible, whilst accepting that in large areas it is not. The merits of a density threshold derived from the asymptotic influence function are shown using both real and simulated data sets. Further, the challenging issue of multivariate bandwidth selection, which is known to be affected detrimentally by sparse data which inevitably arise in higher dimensions, is considered. In an effort to alleviate this problem, two adaptations to generalized cross-validation are implemented, and a simulation study is presented to support the proposed method. It is also discussed how the density threshold and the adapted generalized cross-validation technique introduced herein work neatly together.     

Shrinkage estimators for large covariance matrices in multivariate real and complex normal distributions under an invariant quadratic loss

The problem of estimating large covariance matrices of multivariate real normal and complex normal distributions is considered when the dimension of the variables is larger than the number of samples. The Stein–Haff identities and calculus on eigenstructure for singular Wishart matrices are developed for real and complex cases, respectively. By using these techniques, the unbiased risk estimates for certain classes of estimators for the population covariance matrices under invariant quadratic loss functions are obtained for real and complex cases, respectively. Based on the unbiased risk estimates, shrinkage estimators which are counterparts of the estimators due to Haff [L.R. Haff, Empirical Bayes estimation of the multivariate normal covariance matrix, Ann. Statist. 8 (1980) 586–697] are shown to improve upon the best scalar multiple of the empirical covariance matrix under the invariant quadratic loss functions for both real and complex multivariate normal distributions in the situation where the dimension of the variables is larger than the number of samples.     

On a nonparametric estimator for the finite time survival probability with zero initial surplus

In this paper, we consider the estimation of the finite time survival probability in the classical risk model when the initial surplus is zero. We construct a nonparametric estimator by Fourier inversion and kernel density estimation method. Under some mild assumptions imposed on the kernel, bandwidth and claim size density, we derive the order of the bias and variance, and show that the estimator has asymptotic normality property. Some simulation studies show that the estimator performs quite well in the finite sample setting.