Analysis of ϕ-divergence for loglinear models with constraints under product-multinomial sampling

The loglinear model under product-multinomial sampling with constraints is considered. The asymptotic expansion and normality of the restricted minimum φ-divergence estimator (RMφDE) which is a generalization of the maximum likelihood estimator is presented. Then various statistics based on φ-divergence and RMφDE are used to test various hypothesis test problems under the model considered. These statistics contain the classical loglikelihood ratio test statistics and Pearson chi-squared test statistics. In the last section, a simulation study is implemented.     

Statistical inference in constrained latent class models for multinomial data based on $$\phi $$-divergence measures

In this paper we explore the possibilities of applying \(\phi \)-divergence measures in inferential problems in the field of latent class models (LCMs) for multinomial data. We first treat the problem of estimating the model parameters. As explained below, minimum \(\phi \)-divergence estimators (M\(\phi \)Es) considered in this paper are a natural extension of the maximum likelihood estimator (MLE), the usual estimator for this problem; we study the asymptotic properties of M\(\phi \)Es, showing that they share the same asymptotic distribution as the MLE. To compare the efficiency of the M\(\phi \)Es when the sample size is not big enough to apply the asymptotic results, we have carried out an extensive simulation study; from this study, we conclude that there are estimators in this family that are competitive with the MLE. Next, we deal with the problem of testing whether a LCM for multinomial data fits a data set; again, \(\phi \)-divergence measures can be used to generate a family of test statistics generalizing both the classical likelihood ratio test and the chi-squared test statistics. Finally, we treat the problem of choosing the best model out of a sequence of nested LCMs; as before, \(\phi \)-divergence measures can handle the problem and we derive a family of \(\phi \)-divergence test statistics based on them; we study the asymptotic behavior of these test statistics, showing that it is the same as the classical test statistics. A simulation study for small and moderate sample sizes shows that there are some test statistics in the family that can compete with the classical likelihood ratio and the chi-squared test statistics.     

Minimum <Emphasis Type="Italic">Φ</Emphasis>-Divergence Estimator and <Emphasis Type="Italic">Φ</Emphasis>-Divergence Statistics in Generalized Linear Models with Binary Data

In this paper, we assume that the data are distributed according to a binomial distribution whose probabilities follow a generalized linear model. To fit the data the minimum φ-divergence estimator is studied as a generalization of the maximum likelihood estimator. We use the minimum φ-divergence estimator, which is the basis of some new statistics, for solving the problems of testing in a generalized linear model with binary data. A wide simulation study is carried out for studying the behavior of the new family of estimators as well as of the new family of test statistics. This work was partially supported by Grant MTM2006-06872 and UCM2006-910707.     

Minimum K_{\phi }-divergence estimators for multinomial models and applications

The properties of minimum $K_{\phi }$ -divergence estimators for parametric multinomial populations are well-known when the assumed parametric model is true, namely, they are consistent and asymptotically normally distributed. Here we study these properties when the parametric model is not assumed to be correctly specified. Under certain conditions, these estimators are shown to converge to a well-defined limit and, suitably normalized, they are also asymptotically normal. Two applications of the results obtained are reported. First, two consistent bootstrap estimators of the null distribution of the test statistics in a certain class of goodness-of-fit tests are proposed and studied. Second, two methods for the model selection test problem based on $K_{\phi }$ -divergence type statistics are proposed and studied. Both applications are illustrated with numerical examples.     

The likelihood ratio tests for the dimensionality of regression coefficients

In this paper we give a unified derivation of the likelihood ratio (LR) statistics for testing the hypothesis on the dimensionality of regression coefficients under a usual MANOVA model. We also derive the LR statistics under a general MANOVA model and study their asymptotic null and nonnull distributions. Further it is shown that the test statistic used by Bartlett [4] for testing the hypothesis that the last p?k canonical correlations are all zero is the LR statistic.     

Minimum f-divergence estimators and quasi-likelihood functions

Maximum quasi-likelihood estimators have several nice asymptotic properties. We show that, in many situations, a family of estimators, called the minimum f-divergence estimators, can be defined such that each estimator has the same asymptotic properties as the maximum quasi-likelihood estimator. The family of minimum f-divergence estimators include the maximum quasi-likelihood estimators as a special case. When a quasi-likelihood is the log likelihood from some exponential family, Amari's dual geometries can be used to study the maximum likelihood estimator. A dual geometric structure can also be defined for more general quasi-likelihood functions as well as for the larger family of minimum f-divergence estimators. The relationship between the f-divergence and the quasi-likelihood function and the relationship between the f-divergence and the power divergence is discussed.This work was supported by National Science Foundation grant DMS 88-03584.     

On φ-Families of Probability Distributions

We generalize the exponential family of probability distributions. In our approach, the exponential function is replaced by a φ-function, resulting in a φ-family of probability distributions. We show how φ-families are constructed. In a φ-family, the analogue of the cumulant-generating function is a normalizing function. We define the φ-divergence as the Bregman divergence associated to the normalizing function, providing a generalization of the Kullback–Leibler divergence. A formula for the φ-divergence where the φ-function is the Kaniadakis κ-exponential function is derived.     

A generalized real-valued measure of the inequality associated with a fuzzy random variable

Fuzzy random variables have been introduced by Puri and Ralescu as an extension of random sets. In this paper, we first introduce a real-valued generalized measure of the “relative variation” (or inequality) associated with a fuzzy random variable. This measure is inspired in Csiszár's f-divergence, and extends to fuzzy random variables many well-known inequality indices. To guarantee certain relevant properties of this measure, we have to distinguish two main families of measures which will be characterized. Then, the fundamental properties are derived, and an outstanding measure in each family is separately examined on the basis of an additive decomposition property and an additive decomposability one. Finally, two examples illustrate the application of the study in this paper.     

Log-linear modeling using conditional log-linear structures

Analysis of large dimensional contingency tables is rather difficult. Fienberg and Kim (1999, Journal of American Statistical Association, 94, 229–239) studied the problem of combining conditional (on single variable) log-linear structures for graphical models to obtain partial information about the full graphical log-linear model. In this paper, we consider the general log-linear models and obtain explicit representation for the log-linear parameters of the full model based on that of conditional structures. As a consequence, we give conditions under which a particular log-linear parameter is present or not in the full model. Some of the main results of Fienberg and Kim follow from our results. The explicit relationships between full model and the conditional structures are also presented. The connections between conditional structures and the layer structures are pointed out. We investigate also the hierarchical nature of the full model, based on conditional structures. Kim (2006, Computational Statistics and Data Analysis, 50, 2044–2064) analyzed graphical log-linear models based on conditional log-linear structures, when a set of variables is conditioned. For this case, we employ the Möbius inversion technique to obtain the interaction parameters of the full log-linear model, and discuss their properties. The hierarchical nature of the full model is also studied based on conditional structures. This result could be effectively used for the model selection also. As applications of our results, we have discussed several typical examples, including a real-life example.     

Confidence sets and coverage probabilities based on preliminary estimators in logistic regression models

In this paper we present recentered confidence sets for the parameters of a logistic regression model based on preliminary minimum ?$?$-divergence estimators. Asymptotic coverage probabilities are given as well as a simulation study in order to analyze the coverage probabilities for small and moderate sample sizes.     

Markov measures and extended zeta functions

In this paper we study a family of representations of the Cuntz algebras O _p where p is a prime. These algebras are built on generators and relations. They are C ^?-algebras and their representations are a part of non-commutative harmonic analysis. Starting with specific generators and relations we pass to an ambient C ^?-algebra, for example in one of the Cuntz-algebras. Our representations are motivated by the study of frequency bands in signal processing: We construct induced measures attached to those representations which turned out to be related to a class of zeta functions. For a particular case those measures give rise to a class of Markov measures and q-Bernoulli polynomials. Our approach is amenable to applications in problems from dynamics and mathematical physics: We introduce a deformation parameter q, and an associated family of q-relations where the number q is a ??quantum-deformation,?? and also a parameter in a scale of (Riemann-Ruelle) zeta functions. Our representations are used in turn in a derivation of formulas for this q-zeta function.     

Robust statistical inference based on the C-divergence family

This paper describes a family of divergences, named herein as the C-divergence family, which is a generalized version of the power divergence family and also includes the density power divergence family as a particular member of this class. We explore the connection of this family with other divergence families and establish several characteristics of the corresponding minimum distance estimator including its asymptotic distribution under both discrete and continuous models; we also explore the use of the C-divergence family in parametric tests of hypothesis. We study the influence function of these minimum distance estimators, in both the first and second order, and indicate the possible limitations of the first-order influence function in this case. We also briefly study the breakdown results of the corresponding estimators. Some simulation results and real data examples demonstrate the small sample efficiency and robustness properties of the estimators.

     

Some stochastic $${\varvec{}}$$-divergences in information theory

In this paper, we introduce the concept of stochastic HH-divergences based on convex stochastic processes. As an application, we propose some inequalities related to stochastic HH-divergences for convex stochastic processes. Our result extends HH-divergence in the class of f-divergence to the class of convex stochastic processes.     

Tail dependence between order statistics

In this work, we introduce the s,k-extremal coefficients for studying the tail dependence between the s-th lower and k-th upper order statistics of a normalized random vector. If its margins have tail dependence then so do their order statistics, with the strength of bivariate tail dependence decreasing as two order statistics become farther apart. Some general properties are derived for these dependence measures which can be expressed via copulas of random vectors. Its relations with other extremal dependence measures used in the literature are discussed, such as multivariate tail dependence coefficients, the coefficient η of tail dependence, coefficients based on tail dependence functions, the extremal coefficient ?, the multivariate extremal index and an extremal coefficient for min-stable distributions. Several examples are presented to illustrate the results, including multivariate exponential and multivariate Gumbel distributions widely used in applications.     

Dual divergence estimators and tests: Robustness results

The class of dual ?-divergence estimators (introduced in Broniatowski and Keziou (2009) [5]) is explored with respect to robustness through the influence function approach. For scale and location models, this class is investigated in terms of robustness and asymptotic relative efficiency. Some hypothesis tests based on dual divergence criteria are proposed and their robustness properties are studied. The empirical performances of these estimators and tests are illustrated by Monte Carlo simulation for both non-contaminated and contaminated data.     

Robust empirical optimization is almost the same as mean–variance optimization

We formulate a distributionally robust optimization problem where the deviation of the alternative distribution is controlled by a $?$-divergence penalty in the objective, and show that a large class of these problems are essentially equivalent to a mean–variance problem. We also show that while a “small amount of robustness” always reduces the in-sample expected reward, the reduction in the variance, which is a measure of sensitivity to model misspecification, is an order of magnitude larger.     

Residual empirical processes and their application to GM-testing for the autoregression order

In the paper a new nonparametric generalized M-test for hypotheses about the order of linear autoregression AR(p) is constructed. We also establish robustness of this test in the model of data contamination by independent additive outliers with intensity O(n ^?1/2). Robustness is formulated in terms of limiting power equicontinuity. Test statistics are constructed with the help of residual empirical processes. We establish the asymptotic uniform linearity of these processes in the defined contamination model.     

Higher order optimal approximation of Csiszar's f-divergence

Here are established various sharp and nearly optimal probabilistic inequalities giving the high order approximation of Csiszar's f-divergence between two probability measures, which is the most essential and general tool for their comparison. The above are done through Taylor's formula, generalized Taylor–Widder's formula, an alternative recent expansion formula. Based on these we give many representation formulae of Csiszar's distance, then we estimate in all directions their remainders by using either the norms approach or the modulus of continuity way. Most of the last probabilistic estimates are sharp or nearly sharp, attained by basic simple functions.     

Practical tests for randomized complete block designs

A class of affine-invariant test statistics, including a sign test and a related family of signed-rank tests, is proposed for randomized complete block designs with one observation per treatment. This class is obtained by using the transformation-retransformation approach of Chakraborty, Chaudhuri and Oja along with a directional transformation due to Tyler. Under the minimal assumption of directional symmetry of the underlying distribution, the null asymptotic distribution of the sign test statistic is shown to be chi-square with p-1 degrees of freedom. The same null distribution is also proved for the family of signed-rank statistics under the assumption of symmetry of the underlying distribution. The Pitman asymptotic relative efficiencies of the tests, relative to Hotelling-Hsu's T² are established. Several score functions are discussed including a simple linear score function and the optimal normal score function. The test based on the linear score function is compared to the other members of this family and other statistics in the literature through efficiency calculations and Monte Carlo simulations. This statistic has an excellent performance over a wide range of distributions and for small as well as large dimensions.