Central limit theorem for integrated square error of multivariate nonparametric density estimators

Martingale theory is used to obtain a central limit theorem for degenerate U-statistics with variable kernels, which is applied to derive central limit theorems for the integrated square error of multivariate nonparametric density estimators. Previous approaches to this problem have employed Komlós-Major-Tusnády type approximations to the empiric distribution function, and have required the following two restrictive assumptions which are not necessary using the present approach: (i) the data are in one or two dimensions, and (ii) the estimator is constructed suboptimally.     

Parametric distance functions vs. nonparametric neural networks for estimating road travel distances

Measuring and storing actual road travel distances between the points of a region is often not feasible and it is a common practice to estimate them. The usual approach is to use distance estimators which are parameterized functions of the coordinates of the points. We propose to use nonparametric approaches using neural networks for estimating actual distances. We consider multi-layer perceptrons trained with the back-propagation rule and regression neural networks implementing nonparametric regression using Gaussian kernels. We also consider training multiple estimators and combining them using voting and stacking. On a real-world study using cities drawn from Turkey, we found out that these nonparametric approaches are more accurate than the parametric distance functions. Estimating actual distances has many applications in location and distribution theory.     

Strong consistency of nonparametric Bayes density estimation on compact metric spaces with applications to specific manifolds

This article considers a broad class of kernel mixture density models on compact metric spaces and manifolds. Following a Bayesian approach with a nonparametric prior on the location mixing distribution, sufficient conditions are obtained on the kernel, prior and the underlying space for strong posterior consistency at any continuous density. The prior is also allowed to depend on the sample size n and sufficient conditions are obtained for weak and strong consistency. These conditions are verified on compact Euclidean spaces using multivariate Gaussian kernels, on the hypersphere using a von Mises-Fisher kernel and on the planar shape space using complex Watson kernels.     

The Mode Tree: A Tool for Visualization of Nonparametric Density Features

Abstract

Recognition and extraction of features in a nonparametric density estimate are highly dependent on correct calibration. The data-driven choice of bandwidth h in kernel density estimation is a difficult one that is compounded by the fact that the globally optimal h is not generally optimal for all values of x. In recognition of this fact a new type of graphical tool, the mode tree, is proposed. The basic mode tree plot relates the locations of modes in density estimates with the bandwidths of those estimates. Additional information can be included on the plot indicating factors such as the size of modes, how modes split, and the locations of antimodes and bumps. The use of a mode tree in adaptive multimodality investigations is proposed, and an example is given to show the value in using a normal kernel, as opposed to the biweight or other kernels, in such investigations. Examples of such investigations are provided for Ahrens's chondrite data and van Winkle's Hidalgo stamp data. Finally, the bivariate mode tree is introduced, together with an example using Scott's lipid data.     

Local Linear Estimation of Jump-Diffusion Models by Using Asymmetric Kernels

This article addresses the problem of nonparametric estimation of the first and second infinitesimal moments by using the local linear method of the underlying jump-diffusion models. The motivation behind the study is to use the asymmetric kernels instead of standard kernel smoothing. The basic idea relies on replacing the symmetric kernel by asymmetric kernel and provides a new way of obtaining the nonparametric estimation for jump-diffusion models. We prove that the estimators based on the local linear method for jump-diffusion models are consistent and asymptotically follow normal distribution under the condition of recurrence and stationarity.     

Higher-order accurate polyspectral estimation with flat-top lag-windows

Improved performance in higher-order spectral density estimation is achieved using a general class of infinite-order kernels. These estimates are asymptotically less biased but with the same order of variance as compared to the classical estimators with second-order kernels. A simple, data-dependent algorithm for selecting the bandwidth is introduced and is shown to be consistent with estimating the optimal bandwidth. The combination of the specialized family of kernels with the new bandwidth selection algorithm yields a considerably improved polyspectral estimator surpassing the performances of existing estimators using second-order kernels. Bispectral simulations with several standard models are used to demonstrate the enhanced performance with the proposed methodology.     

Bias Free Threshold Estimation for Jump Intensity Function

In this paper, combining the threshold technique, we reconstruct Nadaraya-Watson estimation using Gamma asymmetric kernels for the unknown jump intensity function of a diffusion process with finite activity jumps. Under mild conditions, we obtain the asymptotic normality for the proposed estimator. Moreover, we have verified the better finite-sampling properties such as bias correction and effciency gains of the underlying estimator compared with other nonparametric estimators through a Monte Carlo experiment.     

On confidence bands for multivariate nonparametric regression

In a multivariate nonparametric regression problem with fixed, deterministic design asymptotic, uniform confidence bands for the regression function are constructed. The construction of the bands is based on the asymptotic distribution of the maximal deviation between a suitable nonparametric estimator and the true regression function which is derived by multivariate strong approximation methods and a limit theorem for the supremum of a stationary Gaussian field over an increasing system of sets. The results are derived for a general class of estimators which includes local polynomial estimators as a special case. The finite sample properties of the proposed asymptotic bands are investigated by means of a small simulation study.     

Multivariate density estimation using dimension reducing information and tail flattening transformations for truncated or censored data

This paper introduces a multivariate density estimator for truncated and censored data with special emphasis on extreme values based on survival analysis. A local constant density estimator is considered. We extend this estimator by means of tail flattening transformation, dimension reducing prior knowledge and a combination of both. The asymptotic theory is derived for the proposed estimators. It shows that the extensions might improve the performance of the density estimator when the transformation and the prior knowledge is not too far away from the true distribution. A simulation study shows that the density estimator based on tail flattening transformation and prior knowledge substantially outperforms the one without prior knowledge, and therefore confirms the asymptotic results. The proposed estimators are illustrated and compared in a data study of fire insurance claims.     

Optimally thresholded realized power variations for Lévy jump diffusion models

Thresholded Realized Power Variations (TPVs) are one of the most popular nonparametric estimators for general continuous-time processes with a wide range of applications. In spite of their popularity, a common drawback lies in the necessity of choosing a suitable threshold for the estimator, an issue which so far has mostly been addressed by heuristic selection methods. To address this important issue, we propose an objective selection method based on desirable optimality properties of the estimators. Concretely, we develop a well-posed optimization problem which, for a fixed sample size and time horizon, selects a threshold that minimizes the expected total number of jump misclassifications committed by the thresholding mechanism associated with these estimators. We analytically solve the optimization problem under mild regularity conditions on the density of the underlying jump distribution, allowing us to provide an explicit infill asymptotic characterization of the resulting optimal thresholding sequence at a fixed time horizon. The leading term of the optimal threshold sequence is shown to be proportional to Lévy’s modulus of continuity of the underlying Brownian motion, hence theoretically justifying and sharpening selection methods previously proposed in the literature based on power functions or multiple testing procedures. Furthermore, building on the aforementioned asymptotic characterization, we develop an estimation algorithm, which allows for a feasible implementation of the newfound optimal sequence. Simulations demonstrate the improved finite sample performance offered by optimal TPV estimators in comparison to other popular non-optimal alternatives.     

Rank-Score Tests in Factorial Designs with Repeated Measures

Nonparametric factorial designs for multivariate observations are considered under the framework of general rank-score statistics. Unlike most of the literature, we do not assume the continuity of the underlying distribution functions. The models studied include general repeated measures designs, compound symmetry designs, and designs for longitudinal data. In particular, designs for ordered categorical data are included. The vectors of the multivariate observations may have different lengths. Moreover, our general framework includes missing values and singular covariance matrices which occur quite frequently in practical data analysis problems. The asymptotic properties of the proposed statistics are studied under general nonparametric hypotheses as well as under a sequence of nonparametric contiguous alternatives. L₂-consistent estimators for the unknown covariance matrices are given and two types of quadratic forms are considered for testing the nonparametric hypotheses. The results are applied to a two-way mixed model assuming compound symmetry and to a factorial design for longitudinal data. The main idea of the proofs is based on some moment inequalities for empirical distribution functions in mixed models. The details are provided in the Appendix.     

Histogram-kernel error and its application for bin width selection in histograms

Histogram and kernel estimators are usually regarded as the two main classical data-based non- parametric tools to estimate the underlying density functions for some given data sets. In this paper we will integrate them and define a histogram-kernel error based on the integrated square error between histogram and binned kernel density estimator, and then exploit its asymptotic properties. Just as indicated in this paper, the histogram-kernel error only depends on the choice of bin width and the data for the given prior kernel densities. The asymptotic optimal bin width is derived by minimizing the mean histogram-kernel error. By comparing with Scott’s optimal bin width formula for a histogram, a new method is proposed to construct the data-based histogram without knowledge of the underlying density function. Monte Carlo study is used to verify the usefulness of our method for different kinds of density functions and sample sizes.     

The nonparametric estimation of long memory spatio-temporal random field models

This paper considers the local linear estimation of a multivariate regression function and its derivatives for a stationary long memory(long range dependent) nonparametric spatio-temporal regression model.Under some mild regularity assumptions, the pointwise strong convergence, the uniform weak consistency with convergence rates and the joint asymptotic distribution of the estimators are established. A simulation study is carried out to illustrate the performance of the proposed estimators.     

扩散模型中扩散系数的非参数组合估计

为了提高扩散系数估计的准确度, 我们利用动态组合时间域与状态域信息提出一个新的组合估计量. 我们发现所提组合估计量能有效估计扩散模型的扩散系数, 正如在本文中模拟所示. 在一定的条件下, 建立了估计量的渐进正态性, 并证明了时间域估计量与状态域估计量是渐进独立的. 大量的模拟展示了所提组合估计量优于单域估计量, 也优于本文所提估计量.     

Oracle inequality for conditional density estimation and an actuarial example

Conditional density estimation in a parametric regression setting, where the problem is to estimate a parametric density of the response given the predictor, is a classical and prominent topic in regression analysis. This article explores this problem in a nonparametric setting where no assumption about shape of an underlying conditional density is made. For the first time in the literature, it is proved that there exists a nonparametric data-driven estimator that matches performance of an oracle which: (i) knows the underlying conditional density, (ii) adapts to an unknown design of predictors, (iii) performs a dimension reduction if the response does not depend on the predictor, (iv) is minimax over a vast set of anisotropic bivariate function classes. All these results are established via an oracle inequality which is on par with ones known in the univariate density estimation literature. Further, the asymptotically optimal estimator is tested on an interesting actuarial example which explores a relationship between credit scoring and premium for basic auto-insurance for 54 undergraduate college students.     

纵向数据混合效应模型的统计分析

本文研究了Tao等人在1999年提出的半参数混合效应模型,在不假设随机效应服从正态分布的条件下,用傅立叶变换的方法构造了随机效应的光滑非参数密度估计,给出了密度估计的公式,研究了其渐近性质,还构造了半参数混合效应模型中参数的估计方法并研究了其大样本性质。     

Generalized partially linear mixed-effects models incorporating mismeasured covariates

In this article we consider a semiparametric generalized mixed-effects model, and propose combining local linear regression, and penalized quasilikelihood and local quasilikelihood techniques to estimate both population and individual parameters and nonparametric curves. The proposed estimators take into account the local correlation structure of the longitudinal data. We establish normality for the estimators of the parameter and asymptotic expansion for the estimators of the nonparametric part. For practical implementation, we propose an appropriate algorithm. We also consider the measurement error problem in covariates in our model, and suggest a strategy for adjusting the effects of measurement errors. We apply the proposed models and methods to study the relation between virologic and immunologic responses in AIDS clinical trials, in which virologic response is classified into binary variables. A dataset from an AIDS clinical study is analyzed.     

Nonparametric additive model-assisted estimation for survey data

An additive model-assisted nonparametric method is investigated to estimate the finite population totals of massive survey data with the aid of auxiliary information. A class of estimators is proposed to improve the precision of the well known Horvitz-Thompson estimators by combining the spline and local polynomial smoothing methods. These estimators are calibrated, asymptotically design-unbiased, consistent, normal and robust in the sense of asymptotically attaining the Godambe-Joshi lower bound to the anticipated variance. A consistent model selection procedure is further developed to select the significant auxiliary variables. The proposed method is sufficiently fast to analyze large survey data of high dimension within seconds. The performance of the proposed method is assessed empirically via simulation studies.     

Estimation of semi-varying coefficient model with surrogate data and validation sampling

In this paper, we investigate the estimation of semi-varying coefficient models when the nonlinear covariates are prone to measurement error. With the help of validation sampling, we propose two estimators of the parameter and the coefficient functions by combining dimension reduction and the profile likelihood methods without any error structure equation specification or error distribution assumption. We establish the asymptotic normality of proposed estimators for both the parametric and nonparametric parts and show that the proposed estimators achieves the best convergence rate. Data-driven bandwidth selection methods are also discussed. Simulations are conducted to evaluate the finite sample property of the estimation methods proposed.     

Estimation, prediction and the Stein phenomenon under divergence loss

We consider two problems: (1) estimate a normal mean under a general divergence loss introduced in [S. Amari, Differential geometry of curved exponential families — curvatures and information loss, Ann. Statist. 10 (1982) 357-387] and [N. Cressie, T.R.C. Read, Multinomial goodness-of-fit tests, J. Roy. Statist. Soc. Ser. B. 46 (1984) 440-464] and (2) find a predictive density of a new observation drawn independently of observations sampled from a normal distribution with the same mean but possibly with a different variance under the same loss. The general divergence loss includes as special cases both the Kullback-Leibler and Bhattacharyya-Hellinger losses. The sample mean, which is a Bayes estimator of the population mean under this loss and the improper uniform prior, is shown to be minimax in any arbitrary dimension. A counterpart of this result for predictive density is also proved in any arbitrary dimension. The admissibility of these rules holds in one dimension, and we conjecture that the result is true in two dimensions as well. However, the general Baranchick [A.J. Baranchick, a family of minimax estimators of the mean of a multivariate normal distribution, Ann. Math. Statist. 41 (1970) 642-645] class of estimators, which includes the James-Stein estimator and the Strawderman [W.E. Strawderman, Proper Bayes minimax estimators of the multivariate normal mean, Ann. Math. Statist. 42 (1971) 385-388] class of estimators, dominates the sample mean in three or higher dimensions for the estimation problem. An analogous class of predictive densities is defined and any member of this class is shown to dominate the predictive density corresponding to a uniform prior in three or higher dimensions. For the prediction problem, in the special case of Kullback-Leibler loss, our results complement to a certain extent some of the recent important work of Komaki [F. Komaki, A shrinkage predictive distribution for multivariate normal observations, Biometrika 88 (2001) 859-864] and George, Liang and Xu [E.I. George, F. Liang, X. Xu, Improved minimax predictive densities under Kullbak-Leibler loss, Ann. Statist. 34 (2006) 78-92]. While our proposed approach produces a general class of predictive densities (not necessarily Bayes, but not excluding Bayes predictors) dominating the predictive density under a uniform prior. We show also that various modifications of the James-Stein estimator continue to dominate the sample mean, and by the duality of estimation and predictive density results which we will show, similar results continue to hold for the prediction problem as well.