Minimum f-divergence estimators and quasi-likelihood functions

Maximum quasi-likelihood estimators have several nice asymptotic properties. We show that, in many situations, a family of estimators, called the minimum f-divergence estimators, can be defined such that each estimator has the same asymptotic properties as the maximum quasi-likelihood estimator. The family of minimum f-divergence estimators include the maximum quasi-likelihood estimators as a special case. When a quasi-likelihood is the log likelihood from some exponential family, Amari's dual geometries can be used to study the maximum likelihood estimator. A dual geometric structure can also be defined for more general quasi-likelihood functions as well as for the larger family of minimum f-divergence estimators. The relationship between the f-divergence and the quasi-likelihood function and the relationship between the f-divergence and the power divergence is discussed.This work was supported by National Science Foundation grant DMS 88-03584.     

Minimum <Emphasis Type="Italic">Φ</Emphasis>-Divergence Estimator and <Emphasis Type="Italic">Φ</Emphasis>-Divergence Statistics in Generalized Linear Models with Binary Data

In this paper, we assume that the data are distributed according to a binomial distribution whose probabilities follow a generalized linear model. To fit the data the minimum φ-divergence estimator is studied as a generalization of the maximum likelihood estimator. We use the minimum φ-divergence estimator, which is the basis of some new statistics, for solving the problems of testing in a generalized linear model with binary data. A wide simulation study is carried out for studying the behavior of the new family of estimators as well as of the new family of test statistics. This work was partially supported by Grant MTM2006-06872 and UCM2006-910707.     

Dual divergence estimators and tests: Robustness results

The class of dual ?-divergence estimators (introduced in Broniatowski and Keziou (2009) [5]) is explored with respect to robustness through the influence function approach. For scale and location models, this class is investigated in terms of robustness and asymptotic relative efficiency. Some hypothesis tests based on dual divergence criteria are proposed and their robustness properties are studied. The empirical performances of these estimators and tests are illustrated by Monte Carlo simulation for both non-contaminated and contaminated data.     

Some general divergence measures for probability distributions

Summary General divergence measures for probability distributions are introduced and their main properties established. Connections with f-divergence corresponding to a convex function fare explored.     

On φ-Families of Probability Distributions

We generalize the exponential family of probability distributions. In our approach, the exponential function is replaced by a φ-function, resulting in a φ-family of probability distributions. We show how φ-families are constructed. In a φ-family, the analogue of the cumulant-generating function is a normalizing function. We define the φ-divergence as the Bregman divergence associated to the normalizing function, providing a generalization of the Kullback–Leibler divergence. A formula for the φ-divergence where the φ-function is the Kaniadakis κ-exponential function is derived.     

Statistical inference in constrained latent class models for multinomial data based on $$\phi $$-divergence measures

In this paper we explore the possibilities of applying \(\phi \)-divergence measures in inferential problems in the field of latent class models (LCMs) for multinomial data. We first treat the problem of estimating the model parameters. As explained below, minimum \(\phi \)-divergence estimators (M\(\phi \)Es) considered in this paper are a natural extension of the maximum likelihood estimator (MLE), the usual estimator for this problem; we study the asymptotic properties of M\(\phi \)Es, showing that they share the same asymptotic distribution as the MLE. To compare the efficiency of the M\(\phi \)Es when the sample size is not big enough to apply the asymptotic results, we have carried out an extensive simulation study; from this study, we conclude that there are estimators in this family that are competitive with the MLE. Next, we deal with the problem of testing whether a LCM for multinomial data fits a data set; again, \(\phi \)-divergence measures can be used to generate a family of test statistics generalizing both the classical likelihood ratio test and the chi-squared test statistics. Finally, we treat the problem of choosing the best model out of a sequence of nested LCMs; as before, \(\phi \)-divergence measures can handle the problem and we derive a family of \(\phi \)-divergence test statistics based on them; we study the asymptotic behavior of these test statistics, showing that it is the same as the classical test statistics. A simulation study for small and moderate sample sizes shows that there are some test statistics in the family that can compete with the classical likelihood ratio and the chi-squared test statistics.     

Weighted L quantile distance estimators for randomly censored data

The asymptotic properties of a family of minimum quantile distance estimators for randomly censored data sets are considered. These procedures produce an estimator of the parameter vector that minimizes a weighted L² distance measure between the Kaplan-Meier quantile function and an assumed parametric family of quantile functions. Regularity conditions are provided which insure that these estimators are consistent and asymptotically normal. An optimal weight function is derived for single parameter families, which, for location/scale families, results in censored sample analogs of estimators such as those suggested by Parzen.     

New families of estimators and test statistics in log-linear models

In this paper we consider categorical data that are distributed according to a multinomial, product-multinomial or Poisson distribution whose expected values follow a log-linear model and we study the inference problem of hypothesis testing in a log-linear model setting. The family of test statistics considered is based on the family of ?-divergence measures. The unknown parameters in the log-linear model under consideration are also estimated using ?-divergence measures: Minimum ?-divergence estimators. A simulation study is included to find test statistics that offer an attractive alternative to the Pearson chi-square and likelihood-ratio test statistics.     

Minimum disparity estimation for continuous models: Efficiency,distributions and robustness

A general class of minimum distance estimators for continuous models called minimum disparity estimators are introduced. The conventional technique is to minimize a distance between a kernel density estimator and the model density. A new approach is introduced here in which the model and the data are smoothed with the same kernel. This makes the methods consistent and asymptotically normal independently of the value of the smoothing parameter; convergence properties of the kernel density estimate are no longer necessary. All the minimum distance estimators considered are shown to be first order efficient provided the kernel is chosen appropriately. Different minimum disparity estimators are compared based on their characterizing residual adjustment function (RAF); this function shows that the robustness features of the estimators can be explained by the shrinkage of certain residuals towards zero. The value of the second derivative of theRAF at zero,A ₂, provides the trade-off between efficiency and robustness. The above properties are demonstrated both by theorems and by simulations.     

Weighted L2 quantile distance estimators for randomly censored data

The asymptotic properties of a family of minimum quantile distance estimators for randomly censored data sets are considered. These procedures produce an estimator of the parameter vector that minimizes a weighted L² distance measure between the Kaplan-Meier quantile function and an assumed parametric family of quantile functions. Regularity conditions are provided which insure that these estimators are consistent and asymptotically normal. An optimal weight function is derived for single parameter families, which, for location/scale families, results in censored sample analogs of estimators such as those suggested by Parzen.     

Asymptotic Normality of the Minimum Non-Hilbertian Distance Estimators for a Diffusion Process with Small Noise

A one-dimensional diffusion type process with small noise is observed up to the time T. It depends on an unknown real parameter. Some minimum distance estimators of this parameter are considered. These estimators are defined using the L ^p-metric or the uniform metric. The limiting distribution of the normalizing minimum distance estimators (as the noise vanishing) is known to be the distribution of a random variable. The distribution of this random variable is studied as the time T goes to the infinity. We will prove under some conditions that it has a limiting Gaussian law. This revised version was published online in June 2006 with corrections to the Cover Date.     

Minimum K_{\phi }-divergence estimators for multinomial models and applications

The properties of minimum $K_{\phi }$ -divergence estimators for parametric multinomial populations are well-known when the assumed parametric model is true, namely, they are consistent and asymptotically normally distributed. Here we study these properties when the parametric model is not assumed to be correctly specified. Under certain conditions, these estimators are shown to converge to a well-defined limit and, suitably normalized, they are also asymptotically normal. Two applications of the results obtained are reported. First, two consistent bootstrap estimators of the null distribution of the test statistics in a certain class of goodness-of-fit tests are proposed and studied. Second, two methods for the model selection test problem based on $K_{\phi }$ -divergence type statistics are proposed and studied. Both applications are illustrated with numerical examples.     

Computing S Estimators for Regression and Multivariate Location/Dispersion

Abstract

An improved resampling algorithm for S estimators reduces the number of times the objective function is evaluated and increases the speed of convergence. With this algorithm, S estimates can be computed in less time than least median squares (LMS) for regression and minimum volume ellipsoid (MVE) for location/scatter estimates with the same accuracy. Here accuracy refers to the randomness due to the algorithm. S estimators are also more statistically efficient than the LMS and MVE estimators, that is, they have less variability due to the randomness of the data.     

Estimating a density and its derivatives via the minimum distance method

Summary This paper deals with minimum distance (MD) estimators and minimum penalized distance (MPD) estimators which are based on the L _p distance. Rates of strong consistency of MPD density estimators are established within the family of density functions which have a bounded m-th derivative. For the case p=2, it is also proved that the MPD density estimator achieves the optimum rate of decrease of the mean integrated square error and the L ₁ error. Estimation of derivatives of the density is considered as well.In a class parametrized by entire functions, it is proved that the rate of convergence of the MD density estimator (and its derivatives) to the unknown density (its derivatives) is of order in expected L ₁ and L ₂ distances. In the same class of distributions, MD estimators of unknown density and its derivatives are proved to achieve an extraordinary rate (log log n/n)^1/2 of strong consistency.     

Confidence sets and coverage probabilities based on preliminary estimators in logistic regression models

In this paper we present recentered confidence sets for the parameters of a logistic regression model based on preliminary minimum ?$?$-divergence estimators. Asymptotic coverage probabilities are given as well as a simulation study in order to analyze the coverage probabilities for small and moderate sample sizes.     

Robust estimation in single-index models when the errors have a unimodal density with unknown nuisance parameter

This paper develops a robust profile estimation method for the parametric and nonparametric components of a single-index model when the errors have a strongly unimodal density with unknown nuisance parameter. We derive consistency results for the link function estimators as well as consistency and asymptotic distribution results for the single-index parameter estimators. Under a log-Gamma model, the sensitivity to anomalous observations is studied using the empirical influence curve. We also discuss a robust K-fold cross-validation procedure to select the smoothing parameters. A numerical study carried on with errors following a log-Gamma model and for contaminated schemes shows the good robustness properties of the proposed estimators and the advantages of considering a robust approach instead of the classical one. A real data set illustrates the use of our proposal.

     

On certain new inequalities in information theory

We prove a global assertion on logarithmic convexity of Csiszár’s ƒ-divergence. It follows that the relative s-information measure is log-convex for s ∈ ℝ, wherefrom some new inequalities connecting Kullback-Leibler divergence and χ² and Hellinger distances arise.     

Dilation to the unilateral shifts

The classical result of Foias says that an operator power dilates to a unilateral shift if and only if it is aC ₀ contraction. In this paper, we consider the corresponding question of dilating to a unilateral shift. We show tht for contractions with at least one defect index finite, dilation and power dilation to some unilateral shift amount to the same thing. The only difference is on the minimum multiplicity of the unilateral shift to which the contraction can be (power) dilated. We also obtain a characterization of contractions which are finite-rank perturbations of a unilateral shift, generalizing the rank-one perturbation result of Nakamura.     

The Dirichlet problem for non‐divergence parabolic equations with discontinuous in time coefficients

We consider the Dirichlet problem for non‐divergence parabolic equation with discontinuous in t coefficients in a half space. The main result is weighted coercive estimates of solutions in anisotropic Sobolev spaces. We give an application of this result to linear and quasi‐linear parabolic equations in a bounded domain. In particular, if the boundary is of class C^1,δ , δ ∈ [0, 1], then we present a coercive estimate of solutions in weighted anisotropic Sobolev spaces, where the weight is a power of the distance to the boundary (© 2009 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)