Real-Time Density and Mode Estimation With Application to Time-Dynamic Mode Tracking

We introduce a nonparametric time-dynamic kernel type density estimate for the situation where an underlying multivariate distribution evolves with time. Based on this time-dynamic density estimate, we propose nonparametric estimates for the time-dynamic mode of the underlying distribution. Our estimators involve boundary kernels for the time dimension so that the estimator is always centered at current time, and multivariate kernels for the spatial dimension of the time-evolving distribution. Under certain mild conditions, the asymptotic behavior of density and mode estimators, especially their uniform convergence in both time and space, is derived. A time-dynamic algorithm for mode tracking is proposed, including automatic bandwidth choices, and is implemented via a mean update algorithm. Simulation studies and real data illustrations demonstrate that the proposed methods work well in practice.     

Data-Adaptive Estimation of Time-Varying Spectral Densities

This article introduces a data-adaptive nonparametric approach for the estimation of time-varying spectral densities from nonstationary time series. Time-varying spectral densities are commonly estimated by local kernel smoothing. The performance of these nonparametric estimators, however, depends crucially on the smoothing bandwidths that need to be specified in both time and frequency direction. As an alternative and extension to traditional bandwidth selection methods, we propose an iterative algorithm for constructing localized smoothing kernels data-adaptively. The main idea, inspired by the concept of propagation-separation, is to determine for a point in the time-frequency plane the largest local vicinity over which smoothing is justified by the data. By shaping the smoothing kernels nonparametrically, our method not only avoids the problem of bandwidth selection in the strict sense but also becomes more flexible. It not only adapts to changing curvature in smoothly varying spectra but also adjusts for structural breaks in the time-varying spectrum. Supplementary materials, including the R package tvspecAdapt containing an implementation of the routine, are available online.     

Data-driven local bandwidth selection for additive models with missing data

This paper deals in the nonparametric estimation of additive models in the presence of missing data in the response variable. Specifically in the case of additive models estimated by the Backfitting algorithm with local polynomial smoothers [1]. Three estimators are presented, one based on the available data and two based on a complete sample from imputation techniques. We also develop a data-driven local bandwidth selector based on a Wild Bootstrap approximation of the mean squared error of the estimators. The performance of the estimators and the local bootstrap bandwidth selection method are explored through simulation experiments.     

Optimal kernel estimation of spot volatility of stochastic differential equations

A unified framework to optimally select the bandwidth and kernel function of spot volatility kernel estimators is put forward. The proposed models include not only classical Brownian motion driven dynamics but also volatility processes that are driven by long-memory fractional Brownian motions or other Gaussian processes. We characterize the leading order terms of the mean squared error, which in turn enables us to determine an explicit formula for the leading term of the optimal bandwidth. Central limit theorems for the estimation error are also obtained. A feasible plug-in type bandwidth selection procedure is then proposed, for which, as a sub-problem, a new estimator of the volatility of volatility is developed. The optimal selection of the kernel function is also investigated. For Brownian Motion type volatilities, the optimal kernel turns out to be an exponential function, while, for fractional Brownian motion type volatilities, easily implementable numerical results to compute the optimal kernels are devised. Simulation studies further confirm the good performance of the proposed methods.     

Non-parametric kernel regression for multinomial data

This paper presents a kernel smoothing method for multinomial regression. A class of estimators of the regression functions is constructed by minimizing a localized power-divergence measure. These estimators include the bandwidth and a single parameter originating in the power-divergence measure as smoothing parameters. An asymptotic theory for the estimators is developed and the bias-adjusted estimators are obtained. A data-based algorithm for selecting the smoothing parameters is also proposed. Simulation results reveal that the proposed algorithm works efficiently.     

Improving estimation in speckled imagery

We propose an analytical bias correction for the maximum likelihood estimators of theG ₁ ⁰ distribution. This distribution is a very powerful tool for speckled imagery analysis, since it is capable of describing a wide range of target roughness. We compare the performance of the corrected estimators with the corresponding original version using Monte Carlo simulation. This second-order bias correction leads to estimators which are better from both the bias and mean square error criteria.     

On the convergence of kernel estimators of probability density functions

Summary The properties of the characteristic function of the fixed-bandwidth kernel estimator of a probability density function are used to derive estimates of the rate of almost sure convergence of such estimators in a family of Hilbert spaces. The convergence of these estimators in a reproducing-kernel Hilbert space is used to prove the uniform convergence of variable-bandwidth estimators. An algorithm employing the fast Fourier transform and heuristic estimates of the optimal bandwidth is presented, and numerical experiments using four density functions are described. This research was supported by the United States Air Force, Air Force Office of Scientific Research, Under Grant Number AFOSR-76-2711.     

Probability Density Function Estimation Using Gamma Kernels

We consider estimating density functions which have support on [0, ) using some gamma probability densities as kernels to replace the fixed and symmetric kernel used in the standard kernel density estimator. The gamma kernels are non-negative and have naturally varying shape. The gamma kernel estimators are free of boundary bias, non-negative and achieve the optimal rate of convergence for the mean integrated squared error. The variance of the gamma kernel estimators at a distance x away from the origin is O(n ^–4/5 x ^–1/2) indicating a smaller variance as x increases. Finite sample comparisons with other boundary bias free kernel estimators are made via simulation to evaluate the performance of the gamma kernel estimators.     

On an improvement of Hill and some other estimators

In the paper, we propose a new idea in the tail-index estimation. This idea allows us to improve the asymptotic performance of the classical Hill estimator and other most popular estimators over the range of the parameters present in the second-order regular-variation condition. We prove the asymptotic normality of the introduced estimators and provide a comparison (using the asymptotic mean-squared error) with other estimators of the tail index.     

On optimal kernel choice for deconvolution

In this note we show that, from a conventional viewpoint, there are particularly close parallels between optimal-kernel-choice problems in non-parametric deconvolution, and their better-understood counterparts in density estimation and regression. However, other aspects of these problems are distinctly different, and this property leads us to conclude that “optimal” kernels do not give satisfactory performance when applied to deconvolution. This unexpected result stems from the fact that standard side conditions, which are used to ensure that the familiar kernel-choice problem has a unique solution, do not have statistically beneficial implications for deconvolution estimators. In consequence, certain “sub-optimal” kernels produce estimators that enjoy both greater efficiency and greater visual smoothness.     

Rate of convergence of a convolution-type estimator of the marginal density of a MA(1) process

In this paper moving-average processes with no parametric assumption on the error distribution are considered. A new convolution-type estimator of the marginal density of a MA(1) is presented. This estimator is closely related to some previous ones used to estimate the integrated squared density and has a structure similar to the ordinary kernel density estimator. For second-order kernels, the rate of convergence of this new estimator is investigated and the rate of the optimal bandwidth obtained. Under limit conditions on the smoothing parameter the convolution-type estimator is proved to be -consistent, which contrasts with the asymptotic behavior of the ordinary kernel density estimator, that is only -consistent.     

Plug-in bandwidth selector for the kernel relative density estimator

This paper is focused on two kernel relative density estimators in a two-sample problem. An asymptotic expression for the mean integrated squared error of these estimators is found and, based on it, two solve- the-equation plug-in bandwidth selectors are proposed. In order to examine their practical performance a simulation study and a practical application to a medical dataset are carried out.     

On progressively first failure censored Lindley distribution

This article deals with the progressively first failure censored Lindley distribution. Maximum likelihood and Bayes estimators of the parameter and reliability characteristics of Lindley distribution based on progressively first failure censored samples are derived. Asymptotic confidence intervals based on observed Fisher information and bootstrap confidence intervals of the parameter are constructed. Bayes estimators using non-informative and gamma informative priors are derived using importance sampling procedure and Metropolis–Hastings (MH) algorithm under squared error loss function. Also, HPD credible intervals based on importance sampling procedure and MH algorithm for the parameter are constructed. To study the performance of various estimators discussed in this article, a Monte Carlo simulation study is conducted. Finally, a real data set is studied for illustration purposes.     

Fuzzy clustering using multiple Gaussian kernels with optimized-parameters

In this paper, we propose a new kernel-based fuzzy clustering algorithm which tries to find the best clustering results using optimal parameters of each kernel in each cluster. It is known that data with nonlinear relationships can be separated using one of the kernel-based fuzzy clustering methods. Two common fuzzy clustering approaches are: clustering with a single kernel and clustering with multiple kernels. While clustering with a single kernel doesn’t work well with “multiple-density” clusters, multiple kernel-based fuzzy clustering tries to find an optimal linear weighted combination of kernels with initial fixed (not necessarily the best) parameters. Our algorithm is an extension of the single kernel-based fuzzy c-means and the multiple kernel-based fuzzy clustering algorithms. In this algorithm, there is no need to give “good” parameters of each kernel and no need to give an initial “good” number of kernels. Every cluster will be characterized by a Gaussian kernel with optimal parameters. In order to show its effective clustering performance, we have compared it to other similar clustering algorithms using different databases and different clustering validity measures.     

Information amount and higher-order efficiency in estimation

By means of second-order asymptotic approximation, the paper clarifies the relationship between the Fisher information of first-order asymptotically efficient estimators and their decision-theoretic performance. It shows that if the estimators are modified so that they have the same asymptotic bias, the information amount can be connected with the risk based on convex loss functions in such a way that the greater information loss of an estimator implies its greater risk. The information loss of the maximum likelihood estimator is shown to be minimal in a general set-up. A multinomial model is used for illustration.     

Stationarity preserving and efficiency increasing probability mass transfers made possible

We develop an efficient computational algorithm that produces efficient Markov chain Monte Carlo (MCMC) transition matrices. The first level of efficiency is measured in terms of the number of operations needed to produce the resulting matrix. The second level of efficiency is evaluated in terms of the asymptotic variance of the resulting MCMC estimators. Results are first given for transition matrices in finite state spaces and then extended to transition kernels in more general settings.     

Asymptotic Properties of Backfitting Estimators

When additive models with more than two covariates are fitted with the backfitting algorithm proposed by Buja et al. [2], the lack of explicit expressions for the estimators makes study of their theoretical properties cumbersome. Recursion provides a convenient way to extend existing theoretical results for bivariate additive models to models of arbitrary dimension. In the case of local polynomial regression smoothers, recursive asymptotic bias and variance expressions for the backfitting estimators are derived. The estimators are shown to achieve the same rate of convergence as those of univariate local polynomial regression. In the case of independence between the covariates, non-recursive bias and variance expressions, as well as the asymptotically optimal values for the bandwidth parameters, are provided.     

Parametric distance functions vs. nonparametric neural networks for estimating road travel distances

Measuring and storing actual road travel distances between the points of a region is often not feasible and it is a common practice to estimate them. The usual approach is to use distance estimators which are parameterized functions of the coordinates of the points. We propose to use nonparametric approaches using neural networks for estimating actual distances. We consider multi-layer perceptrons trained with the back-propagation rule and regression neural networks implementing nonparametric regression using Gaussian kernels. We also consider training multiple estimators and combining them using voting and stacking. On a real-world study using cities drawn from Turkey, we found out that these nonparametric approaches are more accurate than the parametric distance functions. Estimating actual distances has many applications in location and distribution theory.     

Variable bandwidth and one-step local M-estimator

A robust version of local linear regression smoothers augmented with variable bandwidth is studied. The proposed method inherits the advantages of local polynomial regression and overcomes the shortcoming of lack of robustness of least-squares techniques. The use of variable bandwidth enhances the flexibility of the resulting local M-estimators and makes them possible to cope well with spatially inhomogeneous curves, heteroscedastic errors and nonuniform design densities. Under appropriate regularity conditions, it is shown that the proposed estimators exist and are asymptotically normal. Based on the robust estimation equation, one-step local M-estimators are introduced to reduce computational burden. It is demonstrated that the one-step local M-estimators share the same asymptotic distributions as the fully iterative M-estimators, as long as the initial estimators are good enough. In other words, the one-step local M-estimators reduce significantly the computation cost of the fully iterative M-estimators without deteriorating their performance. This fact is also illustrated via simulations.     

Kernel quantile estimator with ICI adaptive bandwidth selection technique

In this article, we consider a class of kernel quantile estimators which is the linear combi- nation of order statistics. This class of kernel quantile estimators can be regarded as an extension of some existing estimators. The exact mean square error expression for this class of estimators will be provided when data are uniformly distributed. The implementation of these estimators depends mostly on the bandwidth selection. We then develop an adaptive method for bandwidth selection based on the intersection confidence intervals （ICI） principle. Monte Carlo studies demonstrate that our proposed approach is comparatively remarkable. We illustrate our method with a real data set.