Data sharpening methods in multivariate local quadratic regression

This paper is concerned with the conditional bias and variance of local quadratic regression to the multivariate predictor variables. Data sharpening methods of nonparametric regression were first proposed by Choi, Hall, Roussion. Recently, a data sharpening estimator of local linear regression was discussed by Naito and Yoshizaki. In this paper, to improve mainly the fitting precision, we extend their results on the asymptotic bias and variance. Using the data sharpening estimator of multivariate local quadratic regression, we are able to derive higher fitting precision. In particular, our approach is simple to implement, since it has an explicit form, and is convenient when analyzing the asymptotic conditional bias and variance of the estimator at the interior and boundary points of the support of the density function.     

A bias corrected nonparametric regression estimator

In this article, we propose a new method of bias reduction in nonparametric regression estimation. The proposed new estimator has asymptotic bias order h⁴, where h is a smoothing parameter, in contrast to the usual bias order h² for the local linear regression. In addition, the proposed estimator has the same order of the asymptotic variance as the local linear regression. Our proposed method is closely related to the bias reduction method for kernel density estimation proposed by Chung and Lindsay (2011). However, our method is not a direct extension of their density estimate, but a totally new one based on the bias cancelation result of their proof.     

A Computationally Efficient Oracle Estimator for Additive Nonparametric Regression with Bootstrap Confidence Intervals

Abstract

This article makes three contributions. First, we introduce a computationally efficient estimator for the component functions in additive nonparametric regression exploiting a different motivation from the marginal integration estimator of Linton and Nielsen. Our method provides a reduction in computation of order n which is highly significant in practice. Second, we define an efficient estimator of the additive components, by inserting the preliminary estimator into a backfitting˙ algorithm but taking one step only, and establish that it is equivalent, in various senses, to the oracle estimator based on knowing the other components. Our two-step estimator is minimax superior to that considered in Opsomer and Ruppert, due to its better bias. Third, we define a bootstrap algorithm for computing pointwise confidence intervals and show that it achieves the correct coverage.     

Optimal subsampling for large-scale quantile regression

To deal with massive data sets, subsampling is known as an effective method which can significantly reduce computational costs in estimating model parameters. In this article, an efficient subsampling method is developed for large-scale quantile regression via Poisson sampling framework, which can solve the memory constraint problem imposed by big data. Under some mild conditions, large sample properties for the estimator involving the weak and strong consistencies, and asymptotic normality are established. Furthermore, the optimal subsampling probabilities are derived according to the A-optimality criterion. It is shown that the estimator based on the optimal subsampling asymptotically achieves a smaller variance than that by the uniform random subsampling. The proposed method is illustrated and evaluated through numerical analyses on both simulated and real data sets.     

Confident estimation for density of a biological population based on line transect sampling

Line transect sampling is a very useful method in survey of wildlife population. Confident interval estimation for density D of a biological population is proposed based on a sequential design. The survey area is occupied by the population whose size is unknown. A stopping rule is proposed by a kernel-based estimator of density function of the perpendicular data at a distance. With this stopping rule, we construct several confidence intervals for D by difference procedures. Some bias reduction techniques are used to modify the confidence intervals. These intervals provide the desired coverage probability as the bandwidth in the stopping rule approaches zero. A simulation study is also given to illustrate the performance of this proposed sequential kernel procedure.     

Nonparametric regression with current status data

We apply nonparametric regression to current status data, which often arises in survival analysis and reliability analysis. While no parametric assumption on the distributions has been imposed, most authors have employed parametric models like linear models to measure the covariate effects on failure times in regression analysis with current status data. We construct a nonparametric estimator of the regression function by modifying the maximum rank correlation (MRC) estimator. Our estimator can deal with the cases where the other estimators do not work. We present the asymptotic bias and the asymptotic distribution of the estimator by adapting a result on equicontinuity of degenerate U-processes to the setup of this paper.     

Estimating the intensity of a cyclic Poisson process in the presence of linear trend

We construct and investigate a consistent kernel-type nonparametric estimator of the intensity function of a cyclic Poisson process in the presence of linear trend. It is assumed that only a single realization of the Poisson process is observed in a bounded window. We prove that the proposed estimator is consistent when the size of the window indefinitely expands. The asymptotic bias, variance, and the mean-squared error of the proposed estimator are also computed. A simulation study shows that the first order asymptotic approximations to the bias and variance of the estimator are not accurate enough. Second order terms for bias and variance were derived in order to be able to predict the numerical results in the simulation. Bias reduction of our estimator is also proposed.     

Nonparametric prediction of stock returns based on yearly data: The long-term view

One of the most studied questions in economics and finance is whether empirical models can be used to predict equity returns or premiums. In this paper, we take the actuarial long-term view and base our prediction on yearly data from 1872 through 2014. While many authors favor the historical mean or other parametric methods, this article focuses on nonlinear relationships between a set of covariates. A bootstrap test on the true functional form of the conditional expected returns confirms that yearly returns on the S&P500 are predictable. The inclusion of prior knowledge in our nonlinear model shows notable improvement in the prediction of excess stock returns compared to a fully nonparametric model. Statistically, a bias and dimension reduction method is proposed to import more structure in the estimation process as an adequate way to circumvent the curse of dimensionality.     

A quantile-copula approach to conditional density estimation

A new kernel-type estimator of the conditional density is proposed. It is based on an efficient quantile transformation of the data. The proposed estimator, which is based on the copula representation, turns out to have a remarkable product form. Its large-sample properties are considered and comparisons in terms of bias and variance are made with competitors based on nonparametric regression. A comparative simulation study is also provided.     

样本函数条件极值中减低偏差的方法

对样本函数条件极值中偏差项的阶进行了分析,探讨了减低偏差项的方法,分析表明古典折刀法、减-d折刀法均不能减低偏差项;在此基础上,提出了减低偏差项的自助法,并论证了在均方误差意义下,θnab是一种较优的估计.     

Reducing asymptotic bias of weak instrumental estimation using independently repeated cross-sectional information

In this paper, we consider the instrumental variable estimation (the two-stage least squares estimator and the limited information maximum likelihood estimator) using weak instruments in a repeated measurements or a panel data model. We show that independently repeated cross-sectional data can reduce the asymptotic bias of the instrumental variable estimation when instruments are weakly correlated with endogenous variables. When the number of repeated measurements tends to infinity, we can achieve consistent instrumental variable estimation with weak instruments.     

Frontier estimation with local polynomials and high power-transformed data

We present a new method for estimating the frontier of a sample. The estimator is based on a local polynomial regression on the power-transformed data. We assume that the exponent of the transformation goes to infinity while the bandwidth goes to zero. We give conditions on these two parameters for obtaining almost complete convergence. The asymptotic conditional bias and variance of the estimator are provided and its good performance is illustrated for some finite sample situations.     

Reduced‐bias kernel estimators of a positive extreme value index

In this paper, we deal with the semi‐parametric estimation of the extreme value index, an important parameter in extreme value analysis. It is well known that many classic estimators, such as the Hill estimator, reveal a strong bias. This problem motivated the study of two classes of kernel estimators. Those classes generalize the classical Hill estimator and have a tuning parameter that enables us to modify the asymptotic mean squared error and eventually to improve their efficiency. Since the improvement in efficiency is not very expressive, we also study new reduced bias estimators based on the two classes of kernel statistics. Under suitable conditions, we prove their asymptotic normality. Moreover, an asymptotic comparison, at optimal levels, shows that the new classes of reduced bias estimators are more efficient than other reduced bias estimator from the literature. An illustration of the finite sample behaviour of the kernel reduced‐bias estimators is also provided through the analysis of a data set in the field of insurance.     

Simple and efficient improvements of multivariate local linear regression

This paper studies improvements of multivariate local linear regression. Two intuitively appealing variance reduction techniques are proposed. They both yield estimators that retain the same asymptotic conditional bias as the multivariate local linear estimator and have smaller asymptotic conditional variances. The estimators are further examined in aspects of bandwidth selection, asymptotic relative efficiency and implementation. Their asymptotic relative efficiencies with respect to the multivariate local linear estimator are very attractive and increase exponentially as the number of covariates increases. Data-driven bandwidth selection procedures for the new estimators are straightforward given those for local linear regression. Since the proposed estimators each has a simple form, implementation is easy and requires much less or about the same amount of effort. In addition, boundary corrections are automatic as in the usual multivariate local linear regression.     

带有辅助信息的ROC曲线两种估计的比较

在医学研究中,常常使用受试者操作特性曲线(ROC)曲线来研究两样本的比较问题。Lloyd构造了ROC曲线的核平滑估计,并给出了其渐近偏差以及渐近标准差。此外,当还可以获悉某一处理组上的辅助信息时,Zhou,Zhou & Ma利用经验似然的方法构造了ROC曲线的核平滑经验似然估计。本文利用"亏量"这个概念比较了带有辅助信息的情况下,对核平滑经验似然估计与完全经验似然估计进行了比较。并给出了核平滑经验似然估计优于完全经验似然估计的结论,并且随着样本容量的增大,该亏量也是无限增大的。     

Tail Index Estimation and an Exponential Regression Model

One of the most important problems involved in the estimation of Pareto indices is the reduction of bias in case the slowly varying part of the Pareto type model disappears at a very slow rate. In other cases, when the bias problem is not so severe, the application of well-known estimators such as the Hill (1975) and the moment estimator (Dekkers et al. (1989)) still asks for an adaptive selection of the sample fraction to be used in such estimation procedures. We show that in both circumstances, solutions can be constructed for the given problems using maximum likelihood estimators based on a regression model for upper order statistics. Via this technique one can also infer about the bias-variance trade-off for a given data set. The behavior of the new maximum likelihood estimator is illustrated through simulation experiments, among others for ARCH processes.     

On a nonparametric estimator for the finite time survival probability with zero initial surplus

In this paper, we consider the estimation of the finite time survival probability in the classical risk model when the initial surplus is zero. We construct a nonparametric estimator by Fourier inversion and kernel density estimation method. Under some mild assumptions imposed on the kernel, bandwidth and claim size density, we derive the order of the bias and variance, and show that the estimator has asymptotic normality property. Some simulation studies show that the estimator performs quite well in the finite sample setting.     

Discovering Sparse Covariance Structures With the Isomap

Regularization of covariance matrices in high dimensions usually either is based on a known ordering of variables or ignores the ordering entirely. This article proposes a method for discovering meaningful orderings of variables based on their correlations using the Isomap, a nonlinear dimension reduction technique designed for manifold embeddings. These orderings are then used to construct a sparse covariance estimator, which is block-diagonal and/or banded. Finding an ordering to which banding can be applied is desirable because banded estimators have been shown to be consistent in high dimensions. We show that in situations where the variables do have such a structure, the Isomap does very well at discovering it, and the resulting regularized estimator performs better for covariance estimation than other regularization methods that ignore variable order, such as thresholding. We also propose a bootstrap approach to constructing the neighborhood graph used by the Isomap, and show it leads to better estimation. We illustrate our method on data on protein consumption, where the variables (food types) have a structure but it cannot be easily described a priori, and on a gene expression dataset. Supplementary materials are available online.     

Estimating and Visualizing Conditional Densities

Abstract

We consider the kernel estimator of conditional density and derive its asymptotic bias, variance, and mean-square error. Optimal bandwidths (with respect to integrated mean-square error) are found and it is shown that the convergence rate of the density estimator is order n ^–2/3. We also note that the conditional mean function obtained from the estimator is equivalent to a kernel smoother. Given the undesirable bias properties of kernel smoothers, we seek a modified conditional density estimator that has mean equivalent to some other nonparametric regression smoother with better bias properties. It is also shown that our modified estimator has smaller mean square error than the standard estimator in some commonly occurring situations. Finally, three graphical methods for visualizing conditional density estimators are discussed and applied to a data set consisting of maximum daily temperatures in Melbourne, Australia.     

Alternatives to a Semi-Parametric Estimator of Parameters of Rare Events—The Jackknife Methodology*

In this paper, and in a context of regularly varying tails, we propose different alternatives to a well-known estimator of the tail index—the Hill estimator (Hill, 1975). These alternatives have essentially in mind a reduction in bias, preferably without increasing Mean Square Error, by the use of suitable Generalized Jackknife methodologies (Gray and Schucany, 1972). The first estimate obtained through this methodolgy is the one introduced by Peng (1998), under a different context. Other Generalized Jackknife estimators are linear combinations of Hill estimators at different levels. This methodology of affine combinations of Hill estimators at different levels may be easily generalized to other semi-parametric estimators of the tail index, like Pickands' estimator (Pickands, 1975) or the Moment's estimator (Dekkers et al., 1989), and consequently to a general real tail index, seeming to be a promising field of research.