Estimation of entropy and other functionals of a multivariate density

For a multivariate density f with respect to Lebesgue measure , the estimation of % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% Waa8qaaeaacaWGkbGaaiikaiaadAgacaGGPaGaamOzaiaadsgacqaH% 8oqBaSqabeqaniabgUIiYdaaaa!4404!\[\int {J(f)fd\mu } \], and in particular % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% Waa8qaaeaacaWGMbWaaWbaaSqabeaacaaIYaaaaOGaamizaiabeY7a% TbWcbeqab0Gaey4kIipaaaa!41E4!\[\int {f^2 d\mu } \] and % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% Waa8qaaeaacaWGMbGaciiBaiaac+gacaGGNbGaamOzaiaadsgacqaH% 8oqBaSqabeqaniabgUIiYdaaaa!44AC!\[\int {f\log fd\mu } \], is studied. These two particular functionals are important in a number of contexts. Asymptotic bias and variance terms are obtained for the estimators % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% WaaybyaeqaleqabaGaey4jIKnaneaacaWGjbaaaOGaeyypa0Zaa8qa% aeaacaWGkbGaaiikamaawagabeWcbeqaaiabgEIizdqdbaGaamOzaa% aakiaacMcacaWGKbGaamOramaaBaaaleaacaWGobaabeaaaeqabeqd% cqGHRiI8aaaa!4994!\[\mathop I\limits^ \wedge = \int {J(\mathop f\limits^ \wedge )dF_N } \] and % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% WaaybyaeqaleqabaGaeSipIOdaneaacaWGjbaaaOGaeyypa0Zaa8qa% aeaacaWGkbGaaiikamaawagabeWcbeqaaiabgEIizdqdbaGaamOzaa% aakiaacMcadaGfGbqabSqabeaacqGHNis2a0qaaiaadAgaaaGccaWG% KbGaeqiVd0galeqabeqdcqGHRiI8aaaa!4C40!\[\mathop I\limits^ \sim = \int {J(\mathop f\limits^ \wedge )\mathop f\limits^ \wedge d\mu } \], where % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% WaaybyaeqaleqabaGaey4jIKnaneaacaWGMbaaaaaa!3E9C!\[{\mathop f\limits^ \wedge }\] is a kernel density estimate of f and F _n is the empirical distribution function based on the random sample X ₁ ,..., X _n from f. For the two functionalsmentioned above, a first order bias term for Î can be made zero by appropriate choices of non-unimodal kernels. Suggestions for the choice of bandwidth are given; for % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% WaaybyaeqaleqabaGaey4jIKnaneaacaWGjbaaaOGaeyypa0Zaa8qa% aeaadaGfGbqabSqabeaacqGHNis2a0qaaiaadAgaaaGccaWGKbGaam% OramaaBaaaleaacaWGobaabeaaaeqabeqdcqGHRiI8aaaa!476C!\[\mathop I\limits^ \wedge = \int {\mathop f\limits^ \wedge dF_N } \], a study of optimal bandwidth is possible.This research was supported by an NSERC Grant and a UBC Killam Research Fellowship.     

Kernel-based adversarial attacks and defenses on support vector classification

While malicious samples are widely found in many application fields of machine learning, suitable countermeasures have been investigated in the field of adversarial machine learning. Due to the importance and popularity of Support Vector Machines (SVMs), we first describe the evasion attack against SVM classification and then propose a defense strategy in this paper. The evasion attack utilizes the classification surface of SVM to iteratively find the minimal perturbations that mislead the nonlinear classifier. Specially, we propose what is called a vulnerability function to measure the vulnerability of the SVM classifiers. Utilizing this vulnerability function, we put forward an effective defense strategy based on the kernel optimization of SVMs with Gaussian kernel against the evasion attack. Our defense method is verified to be very effective on the benchmark datasets, and the SVM classifier becomes more robust after using our kernel optimization scheme.     

Mass shifting roles of negative kernel mass in density estimation

f(x) is a univariate density in C ⁴ with bounded support. For any n and sufficiently small kernel bandwidths, the symmetric appendage of any negative mass, –U, to any smooth unimodal symmetric kernel of order p=2 shifts expected estimator mass from regions where f(x)>0 to regions where f(x)<0. For large n, the mean automatic kernel adaptation induced by –U is analyzed in the simplest MISE reduction scenario: The symmetric appendage of –U to the uniform kernel K(x, X) over MISE-optimal bandwidths reduces MISE by shifting K(x, X) mass asymmetrically across the observation X in the direction of decreasing |f(x)|.     

Optimal convergence properties of kernel density estimators without differentiability conditions

Let X ₁, X ₂, ..., X _n be independent observations from an (unknown) absolutely continuous univariate distribution with density f and let % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% GabmOzayaajaGaaiikaiaadIhacaGGPaGaeyypa0Jaaiikaiaad6ga% caWGObGaaiykamaaCaaaleqabaGaeyOeI0IaaGymaaaakmaaqadaba% Gaam4saiaacUfadaWcgaqaaiaacIcacaWG4bGaeyOeI0Iaamiwamaa% BaaaleaacaWGPbaabeaakiaacMcaaeaacaWGObGaaiyxaaaaaSqaai% aadMgacqGH9aqpcaaIXaaabaGaamOBaaqdcqGHris5aaaa!5356!\[\hat f(x) = (nh)^{ - 1} \sum\nolimits_{i = 1}^n {K[{{(x - X_i )} \mathord{\left/ {\vphantom {{(x - X_i )} {h]}}} \right. \kern-\nulldelimiterspace} {h]}}} \] be a kernel estimator of f(x) at the point x, \s-<x<, with h=h _n (h _n O and nh _n , as n) the bandwidth and K a kernel function of order r. Optimal rates of convergence to zero for the bias and mean square error of such estimators have been studied and established by several authors under varying conditions on K and f. These conditions, however, have invariably included the assumption of existence of the r-th order derivative for f at the point x. It is shown in this paper that these rates of convergence remain valid without any differentiability assumptions on f at x. Instead some simple regularity conditions are imposed on the density f at the point of interest. Our methods are based on certain results in the theory of semi-groups of linear operators and the notions and relations of calculus of finite differences.This research was supported in part by grants from the Natural Sciences and Engineering Research Council of Canada and the University of Alberta Central Research Fund.     

Eigentargets Versus Kernel Eigentargets: Detection of Infrared Point Targets Using Linear and Nonlinear Subspace Algorithms

The Eigentargets method, based on the linear principal component analysis (LPCA), has been used successfully to detect infrared point targets. LPCA is based only on the second-order correlations without taking higher-order statistics into account. That results in the limitation of Eigentargets in target detection. This paper extends Eigentargets, a linear subspace method, to kernel Eigentargets, a detection method based on a nonlinear subspace algorithm. Because the kernel Eigentargets is capable of capturing the part of higher-order statistics, the better detection performance can be achieved. Moreover, the Gaussian intensity model is modified to generate training samples of infrared point targets.     

局部子空间自适应核函数网络

针对目前单一整体核函数网络相对于原始空间特征局部分布缺乏自适应性以及网络规模巨大的不足,本文提出局部子空间自适应核函数网络解决思路。首先将原始空间的高维特征分成一系列低维子空间;然后针对不同子空间构造不同的核函数和核参数局部网络;最后综合所有子空间形成综合核函数网络。同时利用支持向量分布几何意义,选择边界向量代替原始训练样本,大大减少网络的复杂度。由于不同输入子空间构造的核函数以及网络规模相对于样本分布具有一定的自适应性,所以该方法相对于经典全局核函数网络可以得到更好的综合性能。对比实验结果表明本文提出的方法能够改进高维模式识别的性能,是一种值得实际应用推广的方法。     

A Kernel PCA Shape Prior and Edge Based MRF Image Segmentation

We introduce both shape prior and edge information to Markov random field (MRF) to segment target of interest in images.Kernel Principal component analysis (PCA) is performed on a set of training shapes to obtain statistical shape representation.Edges are extracted directly from images.Both of them are added to the MRF energy function and the integrated energy function is minimized by graph cuts.An alignment procedure is presented to deal with variations between the target object and shape templates.Edge information makes the influence of inaccurate shape alignment not too severe,and brings result smoother.The experiments indicate that shape and edge play important roles for complete and robust foreground segmentation.     

A new SVM-based relevance feedback image retrieval using probabilistic feature and weighted kernel function

Relevance feedback (RF) is an effective approach to bridge the gap between low-level visual features and high-level semantic meanings in content-based image retrieval (CBIR). The support vector machine (SVM) based RF mechanisms have been used in different fields of image retrieval, but they often treat all positive and negative feedback samples equally, which will inevitably degrade the effectiveness of SVM-based RF approaches for CBIR. In fact, positive and negative feedback samples, different positive feedback samples, and different negative feedback samples all always have distinct properties. Moreover, each feedback interaction process is usually tedious and time-consuming because of complex visual features, so if too many times of iteration of feedback are asked, users may be impatient to interact with the CBIR system. To overcome the above limitations, we propose a new SVM-based RF approach using probabilistic feature and weighted kernel function in this paper. Firstly, the probabilistic features of each image are extracted by using principal components analysis (PCA) and the adapted Gaussian mixture models (AGMM) based dimension reduction, and the similarity is computed by employing Kullback–Leibler divergence. Secondly, the positive feedback samples and negative feedback samples are marked, and all feedback samples’ weight values are computed by utilizing the samples-based Relief feature weighting. Finally, the SVM kernel function is modified dynamically according to the feedback samples’ weight values. Extensive simulations on large databases show that the proposed algorithm is significantly more effective than the state-of-the-art approaches.     

矿产品水分含量代表值Bootstrap模拟取样估计

在对矿产品水分含量基础统计学特征描述的基础上,采用内核密度估计对水分含量数据多态性进行了分析,根据双态分布的特点,使用Bootstrap模拟取样方法对试验样本值模拟重复取样,以多次Bootstrap模拟取样的均值与标准偏差作为矿产品水分含量有限样本代表值及标准偏差的稳健估计,实践证明Bootstrap模拟取样估计对矿产品水分含量代表值的估计是有效的,该项研究为矿产品水分含量代表值的准确评估提供了一种新方法.     

农残水平测试中数据多态性分析及指定值估计研究

对全国农残水平测试中毒死蜱、氯氰菊酯数据进行统计分析,在数据统计分布特征研究基础上,使用内核密度估计进行数据多态性分析,使用bootstrap模拟取样法对数据样本值重复取样,以获得稳健的水平测试样品待测物含量代表值估计、标准误差及置信区间描述,证明以bootstrap模拟取样法获取的均值与标准偏差作为有限单次样本代表值是合理、有效的,解决了四分位稳健统计方法对非正态多态分布代表值估计不稳定问题及取样理论中取样样本数限制的瓶颈,为能力验证计划指定值的获取提供了一种新方法。