Boundary kernels for adaptive density estimators on regions with irregular boundaries

In some applications of kernel density estimation the data may have a highly non-uniform distribution and be confined to a compact region. Standard fixed bandwidth density estimates can struggle to cope with the spatially variable smoothing requirements, and will be subject to excessive bias at the boundary of the region. While adaptive kernel estimators can address the first of these issues, the study of boundary kernel methods has been restricted to the fixed bandwidth context. We propose a new linear boundary kernel which reduces the asymptotic order of the bias of an adaptive density estimator at the boundary, and is simple to implement even on an irregular boundary. The properties of this adaptive boundary kernel are examined theoretically. In particular, we demonstrate that the asymptotic performance of the density estimator is maintained when the adaptive bandwidth is defined in terms of a pilot estimate rather than the true underlying density. We examine the performance for finite sample sizes numerically through analysis of simulated and real data sets.     

A robust estimator of multivariate location based on projection

This paper presents an estimator of location vector based on one-dimensional projection of high dimensional data. The properties of the new estimator including consistency ,asymptotic normality and robustness are discussed. It is proved that the estimator is not only stronglyconsistent and asymptotically normal but also with a breakdown point 1/2 and a bounded influence function.     

Robust parameter estimation with a small bias against heavy contamination

In this paper we consider robust parameter estimation based on a certain cross entropy and divergence. The robust estimate is defined as the minimizer of the empirically estimated cross entropy. It is shown that the robust estimate can be regarded as a kind of projection from the viewpoint of a Pythagorean relation based on the divergence. This property implies that the bias caused by outliers can become sufficiently small even in the case of heavy contamination. It is seen that the asymptotic variance of the robust estimator is naturally overweighted in proportion to the ratio of contamination. One may surmise that another form of cross entropy can present the same behavior as that discussed above. It can be proved under some conditions that no cross entropy can present the same behavior except for the cross entropy considered here and its monotone transformation.     

Asymptotic properties of conditional maximum likelihood estimator in a certain exponential model

The conditional maximum likelihood estimator is suggested as an alternative to the maximum likelihood estimator and is favorable for an estimator of a dispersion parameter in the normal distribution, the inverse-Gaussian distribution, and so on. However, it is not clear whether the conditional maximum likelihood estimator is asymptotically efficient in general. Consider the case where it is asymptotically efficient and its asymptotic covariance depends only on an objective parameter in an exponential model. This remand implies that the exponential model possesses a certain parallel foliation. In this situation, this paper investigates asymptotic properties of the conditional maximum likelihood estimator and compares the conditional maximum likelihood estimator with the maximum likelihood estimator. We see that the bias of the former is more robust than that of the latter and that two estimators are very close, especially in the sense of bias-corrected version. The mean Pythagorean relation is also discussed.     

Data sharpening methods in multivariate local quadratic regression

This paper is concerned with the conditional bias and variance of local quadratic regression to the multivariate predictor variables. Data sharpening methods of nonparametric regression were first proposed by Choi, Hall, Roussion. Recently, a data sharpening estimator of local linear regression was discussed by Naito and Yoshizaki. In this paper, to improve mainly the fitting precision, we extend their results on the asymptotic bias and variance. Using the data sharpening estimator of multivariate local quadratic regression, we are able to derive higher fitting precision. In particular, our approach is simple to implement, since it has an explicit form, and is convenient when analyzing the asymptotic conditional bias and variance of the estimator at the interior and boundary points of the support of the density function.     

Quasi-maximum likelihood estimation in GARCH processes when some coefficients are equal to zero

The asymptotic distribution of the quasi-maximum likelihood (QML) estimator is established for generalized autoregressive conditional heteroskedastic (GARCH) processes, when the true parameter may have zero coefficients. This asymptotic distribution is the projection of a normal vector distribution onto a convex cone. The results are derived under mild conditions. For an important subclass of models, no moment condition is imposed on the GARCH process. The main practical implication of these results concerns the estimation of overidentified GARCH models.     

A comparison of minimax and least squares estimators in linear regression with polyhedral prior information

We consider the linear regression model where prior information in the form of linear inequalities restricts the parameter space to a polyhedron. Since the linear minimax estimator has, in general, to be determined numerically, it was proposed to minimize an upper bound of the maximum risk instead. The resulting so-called quasiminimax estimator can be easily calculated in closed form. Unfortunately, both minimax estimators may violate the prior information. Therefore, we consider projection estimators which are obtained by projecting the estimate in an optional second step. The performance of these estimators is investigated in a Monte Carlo study together with several least squares estimators, including the inequality restricted least squares estimator. It turns out that both the projected and the unprojected quasiminimax estimators have the best average performance.     

Asymptotics of Bayesian median loss estimation

We establish the consistency, asymptotic normality, and efficiency for estimators derived by minimizing the median of a loss function in a Bayesian context. We contrast this procedure with the behavior of two Frequentist procedures, the least median of squares (LMS) and the least trimmed squares (LTS) estimators, in regression problems. The LMS estimator is the Frequentist version of our estimator, and the LTS estimator approaches a median-based estimator as the trimming approaches 50% on each side. We argue that the Bayesian median-based method is a good tradeoff between the two Frequentist estimators.     

Simple and efficient improvements of multivariate local linear regression

This paper studies improvements of multivariate local linear regression. Two intuitively appealing variance reduction techniques are proposed. They both yield estimators that retain the same asymptotic conditional bias as the multivariate local linear estimator and have smaller asymptotic conditional variances. The estimators are further examined in aspects of bandwidth selection, asymptotic relative efficiency and implementation. Their asymptotic relative efficiencies with respect to the multivariate local linear estimator are very attractive and increase exponentially as the number of covariates increases. Data-driven bandwidth selection procedures for the new estimators are straightforward given those for local linear regression. Since the proposed estimators each has a simple form, implementation is easy and requires much less or about the same amount of effort. In addition, boundary corrections are automatic as in the usual multivariate local linear regression.     

Minimax adjustment technique in a parameter restricted linear model

We consider an approach yielding a minimax estimator in the linear regression model with a priori information on the parameter vector, e.g., ellipsoidal restrictions. This estimator is computed directly from the loss function and can be motivated by the general Pitman nearness criterion. It turns out that this approach coincides with the projection estimator which is obtained by projecting an initial arbitrary estimate on the subset defined by the restrictions.     

A robust and efficient adaptive reweighted estimator of multivariate location and scatter

This article proposes a reweighted estimator of multivariate location and scatter, with weights adaptively computed from the data. Its breakdown point and asymptotic behavior under elliptical distributions are established. This adaptive estimator is able to attain simultaneously the maximum possible breakdown point for affine equivariant estimators and full asymptotic efficiency at the multivariate normal distribution. For the special case of hard-rejection weights and the MCD as initial estimator, it is shown to be more efficient than its non-adaptive counterpart for a broad range of heavy-tailed elliptical distributions. A Monte Carlo study shows that the adaptive estimator is as robust as its non-adaptive relative for several types of bias-inducing contaminations, while it is remarkably more efficient under normality for sample sizes as small as 200.     

Change point estimators by local polynomial fits under a dependence assumption

We study a random design regression model generated by dependent observations, when the regression function itself (or its ν-th derivative) may have a change or discontinuity point. A method based on the local polynomial fits with one-sided kernels to estimate the location and the jump size of the change point is applied in this paper. When the jump location is known, a central limit theorem for the estimator of the jump size is established; when the jump location is unknown, we first obtain a functional limit theorem for a local dilated-rescaled version estimator of the jump size and then give the asymptotic distributions for the estimators of the location and the jump size of the change point. The asymptotic results obtained in this paper can be viewed as extensions of corresponding results for independent observations. Furthermore, a simulated example is given to show that our theory and method perform well in practice.     

Cross-validated bagged learning

Many applications aim to learn a high dimensional parameter of a data generating distribution based on a sample of independent and identically distributed observations. For example, the goal might be to estimate the conditional mean of an outcome given a list of input variables. In this prediction context, bootstrap aggregating (bagging) has been introduced as a method to reduce the variance of a given estimator at little cost to bias. Bagging involves applying an estimator to multiple bootstrap samples and averaging the result across bootstrap samples. In order to address the curse of dimensionality, a common practice has been to apply bagging to estimators which themselves use cross-validation, thereby using cross-validation within a bootstrap sample to select fine-tuning parameters trading off bias and variance of the bootstrap sample-specific candidate estimators. In this article we point out that in order to achieve the correct bias variance trade-off for the parameter of interest, one should apply the cross-validation selector externally to candidate bagged estimators indexed by these fine-tuning parameters. We use three simulations to compare the new cross-validated bagging method with bagging of cross-validated estimators and bagging of non-cross-validated estimators.     

Asymptotic bias and variance for a general class of varying bandwidth density estimators

Summary We consider a general class of varying bandwidth estimators of a probability density function. The class includes the Abramson estimator, transformation kernel density estimator (TKDE), Jones transformation kernel density estimator (JTKDE), nearest neighbour type estimator (NN), Jones-Linton-Nielsen estimator (JLN), Taylor series approximations of TKDE (TTKDE) and Simpson's formula approximations of TKDE (STKDE). Each of these estimators needs a pilot estimator. Starting with an ordinary kernel estimator , it is possible to iterate and compute a sequence of estimates , using each estimate as a pilot estimator in the next step. The first main result is a formula for the bias order. If the bandwidths used in different steps have a common orderh=h(n), the bias of is of orderh ^2km ,k=1, ...,t. Hereh ^m is the bias order of the ideal estimator (defined by using the unknownf as pilot). The second main result is a recursive formula for the leading bias and stochastic terms in an asymptotic expansion of the density estimates. Ifm<, it is possible to make asymptotically equivalent to the ideal estimator.     

Parametric tail copula estimation and model testing

Parametric models for tail copulas are being used for modeling tail dependence and maximum likelihood estimation is employed to estimate unknown parameters. However, two important questions seem unanswered in the literature: (1) What is the asymptotic distribution of the MLE and (2) how does one test the parametric model? In this paper, we answer these two questions in the case of a single parameter for ease of illustration. A simulation study is provided to investigate the finite sample performance of the proposed estimator and test.     

Statistical properties of a kernel-type estimator of the intensity function of a cyclic Poisson process

We consider a kernel-type nonparametric estimator of the intensity function of a cyclic Poisson process when the period is unknown. We assume that only a single realization of the Poisson process is observed in a bounded window which expands in time. We compute the asymptotic bias, variance, and the mean-squared error of the estimator when the window indefinitely expands.     

Cramér-von Mises type estimation of the regression parameter: The rank analogue

A point estimator based on minimization of the rank analogue of the Cramér-von Mises statistic is proposed for the slope parameter β in the simple linear regression model. The asymptotic distribution of the estimator is derived and its variance is compared to the asymptotic variances of several common estimators for β at various underlying distributions.     

Simultaneous estimation of Poisson means

In the simultaneous estimation of means from independent Poisson distributions, an estimator is developed which incorporates a prior mean and variance for each Poisson mean estimated. This estimator possesses substantially smaller risk than the usual estimator in a region of the parameter space and seems superior to other estimators proposed to estimate p Poisson means. It is indicated through two asymptotic results that, unlike the conjugate Bayes estimator, the risk of the estimator does not greatly exceed the risk of the usual estimator outside of the region of risk improvement.     

Frontier estimation via kernel regression on high power-transformed data

We present a new method for estimating the frontier of a multidimensional sample. The estimator is based on a kernel regression on the power-transformed data. We assume that the exponent of the transformation goes to infinity while the bandwidth of the kernel goes to zero. We give conditions on these two parameters to obtain complete convergence and asymptotic normality. The good performance of the estimator is illustrated on some finite sample situations.     

Functional-coefficient partially linear regression model

In this paper, the functional-coefficient partially linear regression (FCPLR) model is proposed by combining nonparametric and functional-coefficient regression (FCR) model. It includes the FCR model and the nonparametric regression (NPR) model as its special cases. It is also a generalization of the partially linear regression (PLR) model obtained by replacing the parameters in the PLR model with some functions of the covariates. The local linear technique and the integrated method are employed to give initial estimators of all functions in the FCPLR model. These initial estimators are asymptotically normal. The initial estimator of the constant part function shares the same bias as the local linear estimator of this function in the univariate nonparametric model, but the variance of the former is bigger than that of the latter. Similarly, initial estimators of every coefficient function share the same bias as the local linear estimates in the univariate FCR model, but the variance of the former is bigger than that of the latter. To decrease the variance of the initial estimates, a one-step back-fitting technique is used to obtain the improved estimators of all functions. The improved estimator of the constant part function has the same asymptotic normality property as the local linear nonparametric regression for univariate data. The improved estimators of the coefficient functions have the same asymptotic normality properties as the local linear estimates in FCR model. The bandwidths and the smoothing variables are selected by a data-driven method. Both simulated and real data examples related to nonlinear time series modeling are used to illustrate the applications of the FCPLR model.