A novel distribution classifier

We present a novel classifier for a collection of nonnegative L¹ functions. Given two sets of data, one set coming from “similar” distributions labeled as normal, and the other unspecified labeled as abnormal. To understand the structure of normality, and further to classify new data with minimal errors, we propose to find the smallest CKL spheres (based on Csiszar divergences) including as many normal data as possible and excluding as many abnormal data as possible. We prove the existence and uniqueness of such a classifier.     

An information-theoretic framework for robustness

This is a paper about the foundation of robust inference. As a specific example, we consider semiparametric location models that involve a shape parameter. We argue that robust methods result via the selection of a representative shape from a set of allowable shapes. To perform this selection, we need a measure of disparity between the true shape and the shape to be used in the inference. Given such a disparity, we propose to solve a certain minimax problem. The paper discusses in detail the use of the Kullback-Leibler divergence for the selection of shapes. The resulting estimators are shown to have redescending influence functions when the set of allowable shapes contains heavy-tailed members. The paper closes with a brief discussion of the next logical step, namely the representation of a set of shapes by a pair of selected shapes.     

A characterization of second order efficiency in a curved exponential family

Summary The optimality of estimation method is investigated in a curved exponential family. A risk function, which is an extension of a residual sum of squares in regression analysis, is introduced. It is shown that second order efficiency of an estimation method is equivalent to attain the minimum among limiting risks of all estimation methods.     

Shannon-information entropy sum in the confined hydrogenic atom

The appearance of critical points in the Shannon entropy sum as a function of confinement radius, in ground and excited state confined hydrogenic systems, is discussed. We illustrate that the Coulomb potential in tandem with the hard sphere confinement are responsible for these points. The positions of these points are observed to vary with the intensity of the potential. The effects of the Coulomb potential on the system are further probed, by examining the differences between the densities of the confined atom and those of the particle confined in a spherical box, for the same confinement radius. These differences are quantified by using Kullback-Leibler and cumulative residual Kullback-Leibler distance measures from information theory. These measures detect that the effects of the Coulomb potential are squeezed out of the system as the confinement radius decreases. That is, the confined atom densities resemble the particle in a box ones, for smaller confinement radii. Furthermore, the critical points in the entropy sum lie in the same regions where there are changes in the distance measures, as the atom behaves more particle in a spherical box-like. The analysis is further complemented by examination of the derivative of the entropy sum with respect to confinement radius. This study illustrates the inhomogeneity in the magnitudes of the derivatives of the entropy sum components and their dependence on the Coulomb potential. A link between the derivative and the entropic force is also illustrated and discussed. Similar behaviors are observed when the virial ratio is compared to the entropic power one, as a function of confinement radius.     

Convergence rate for predictive recursion estimation of finite mixtures

Predictive recursion (PR) is a fast stochastic algorithm for nonparametric estimation of mixing distributions in mixture models. It is known that the PR estimates of both the mixing and mixture densities are consistent under fairly mild conditions, but currently very little is known about the rate of convergence. Here I first investigate asymptotic convergence properties of the PR estimate under model misspecification in the special case of finite mixtures with known support. Tools from stochastic approximation theory are used to prove that the PR estimates converge, to the best Kullback-Leibler approximation, at a nearly root-n rate. When the support is unknown, PR can be used to construct an objective function which, when optimized, yields an estimate of the support. I apply the known-support results to derive a rate of convergence for this modified PR estimate in the unknown support case, which compares favorably to known optimal rates.     

关于不适定积分方程修正OS-EM算法的收敛性研究

主要针对修正OS-EM(Ordered-Subset Expectation-Maximization)重建算法进行研究, 即利用超松弛参数来加速有序子集, 期望最大化的快速重建算法, 并且通过OS-EM算法来进行收敛性分析. 此外, 还充分利用KL距离的一些性质, 以探究在精确数据的情况下, 修正OS-EM算法的单调性及其方程解的收敛性.     

矩阵F分布渐近正态分布的一种方式(英文)

本文主要讨论矩阵F分布的一致渐近正态性.通过计算矩阵F分布和多元正态分布的Kullback-Leibler距离,找到了矩阵F分布一致渐近正态分布的条件.     

Minimaxity and nonminimaxity of a preliminary test estimator for the multivariate normal mean

Summary The problem is to estimate the mean of ap-dimensional normal distribution in the situation where there is vague information that the mean vector might be equal to zero vector. Minimax property of the preliminary test estimator obtained by the use of AIC (Akaike's Information Criterion) procedure is discussed under a loss function which is based on Kullback-Leibler information measure and evaluates both an error of model selection and that of estimation. Whenp is even, the minimaxity is shown to hold for small values ofp but not for large values.     

Semiparametric and Additive Model Selection Using an Improved Akaike Information Criterion

Abstract

An improved AIC-based criterion is derived for model selection in general smoothing-based modeling, including semiparametric models and additive models. Examples are provided of applications to goodness-of-fit, smoothing parameter and variable selection in an additive model and semiparametric models, and variable selection in a model with a nonlinear function of linear terms.     

Structured nonnegative matrix factorization with applications to hidden Markov realization and clustering

In this paper, we study the structured nonnegative matrix factorization problem: given a square, nonnegative matrix P, decompose it as P=VAV^? with V and A nonnegative matrices and with the dimension of A as small as possible. We propose an iterative approach that minimizes the Kullback-Leibler divergence between P and VAV^? subject to the nonnegativity constraints on A and V with the dimension of A given. The approximate structured decomposition P?VAV^? is closely related to the approximate symmetric decomposition P?VV^?. It is shown that the approach for finding an approximate structured decomposition can be adapted to solve the symmetric decomposition problem approximately. Finally, we apply the nonnegative decomposition VAV^? to the hidden Markov realization problem and to the clustering of data vectors based on their distance matrix.