Robust discrimination under a hierarchy on the scatter matrices

Under normality, Flury and Schmid [Quadratic discriminant functions with constraints on the covariances matrices: some asymptotic results, J. Multivariate Anal. 40 (1992) 244-261] investigated the asymptotic properties of the quadratic discrimination procedure under hierarchical models for the scatter matrices, that is: (i) arbitrary scatter matrices, (ii) common principal components, (iii) proportional scatter matrices and (iv) identical matrices. In this paper, we study the properties of robust quadratic discrimination rules based on robust estimates of the involved parameters. Our analysis is based on the partial influence functions of the functionals related to these parameters and allows to derive the asymptotic variances of the estimated coefficients under models (i)-(iv). From them, we conclude that the asymptotic variances verify the same order relations as those obtained by Flury and Schmid [Quadratic discriminant functions with constraints on the covariances matrices: some asymptotic results, J. Multivariate Anal. 40 (1992) 244-261] for the classical estimators. We also perform a Monte Carlo study for different sample sizes and different hierarchies which shows the advantage of using robust procedures over classical ones, when anomalous data are present. It also confirms that better rates of misclassification can be achieved if a more parsimonious model among all the correct ones is used instead of the standard quadratic discrimination.     

High breakdown mixture discriminant analysis

Robust S-estimation is proposed for multivariate Gaussian mixture models generalizing the work of Hastie and Tibshirani (J. Roy. Statist. Soc. Ser. B 58 (1996) 155). In the case of Gaussian Mixture models, the unknown location and scale parameters are estimated by the EM algorithm. In the presence of outliers, the maximum likelihood estimators of the unknown parameters are affected, resulting in the misclassification of the observations. The robust S-estimators of the unknown parameters replace the non-robust estimators from M-step of the EM algorithm. The results were compared with the standard mixture discriminant analysis approach using the probability of misclassification criterion. This comparison showed a slight reduction in the average probability of misclassification using robust S-estimators as compared to the standard maximum likelihood estimators.     

The one- and multi-sample problem for functional data with application to projective shape analysis

In this paper tests are derived for testing neighborhood hypotheses for the one- and multi-sample problem for functional data. Our methodology is used to generalize testing in projective shape analysis, which has traditionally involving data consisting of finite number of points, to the functional case. The one-sample test is applied to the problem of scene identification, in the context of the projective shape of a planar curve.     

Robustness of A-optimal designs

Suppose that Y=(Y_i) is a normal random vector with mean Xb and covariance σ²I_n, where b is a p-dimensional vector (b_j),X=(X_ij) is an n×p matrix. A-optimal designs X are chosen from the traditional set D of A-optimal designs for ρ=0 such that X is still A-optimal in D when the components Y_i are dependent, i.e., for i≠i′, the covariance of Y_i,Y_i^′ is ρ with ρ≠0. Such designs depend on the sign of ρ. The general results are applied to X=(X_ij), where X_ij∈{-1,1}; this corresponds to a factorial design with -1,1 representing low level or high level respectively, or corresponds to a weighing design with -1,1 representing an object j with weight b_j being weighed on the left and right of a chemical balance respectively.     

Order restricted inference for sequential k-out-of-n systems

Sequential order statistics have been introduced to model sequential k-out-of-n systems which, as an extension of k-out-of-n systems, allow the failure of some components of the system to influence the remaining ones. Based on an independent sample of vectors of sequential order statistics, the maximum likelihood estimators of the model parameters of a sequential k-out-of-n system are derived under order restrictions. Special attention is paid to the simultaneous maximum likelihood estimation of the model parameters and the distribution parameters for a flexible location-scale family. Furthermore, order restricted hypothesis tests are considered for making the decision whether the usual k-out-of-n model or the general sequential k-out-of-n model is appropriate for a given data.     

Dual divergence estimators and tests: Robustness results

The class of dual ?-divergence estimators (introduced in Broniatowski and Keziou (2009) [5]) is explored with respect to robustness through the influence function approach. For scale and location models, this class is investigated in terms of robustness and asymptotic relative efficiency. Some hypothesis tests based on dual divergence criteria are proposed and their robustness properties are studied. The empirical performances of these estimators and tests are illustrated by Monte Carlo simulation for both non-contaminated and contaminated data.     

Minimum distance classification rules for high dimensional data

In this article, the problem of classifying a new observation vector into one of the two known groups Π_i,i=1,2, distributed as multivariate normal with common covariance matrix is considered. The total number of observation vectors from the two groups is, however, less than the dimension of the observation vectors. A sample-squared distance between the two groups, using Moore-Penrose inverse, is introduced. A classification rule based on the minimum distance is proposed to classify an observation vector into two or several groups. An expression for the error of misclassification when there are only two groups is derived for large p and n=O(p^δ),0<δ<1.     

Estimation of means of multivariate normal populations with order restriction

Multivariate isotonic regression theory plays a key role in the field of statistical inference under order restriction for vector valued parameters. Two cases of estimating multivariate normal means under order restricted set are considered. One case is that covariance matrices are known, the other one is that covariance matrices are unknown but are restricted by partial order. This paper shows that when covariance matrices are known, the estimator given by this paper always dominates unrestricted maximum likelihood estimator uniformly, and when covariance matrices are unknown, the plug-in estimator dominates unrestricted maximum likelihood estimator under the order restricted set of covariance matrices. The isotonic regression estimators in this paper are the generalizations of plug-in estimators in unitary case.     

Infinitesimal bendings of high orders for homogeneous surfaces with positive curvature and a flat point

For any given n∈Z⁺ and k∈Z⁺, infinitesimal bendings of order n and of class Ck are constructed for a class of surfaces in R³. These surfaces are given as graphs of homogeneous polynomials P(x,y) and they have positive curvature except at the origin.     

Finite sample tail behavior of multivariate location estimators

A finite sample performance measure of multivariate location estimators is introduced based on “tail behavior”. The tail performance of multivariate “monotone” location estimators and the halfspace depth based “non-monotone” location estimators including the Tukey halfspace median and multivariate L-estimators is investigated. The connections among the finite sample performance measure, the finite sample breakdown point, and the halfspace depth are revealed. It turns out that estimators with high breakdown point or halfspace depth have “appealing” tail performance. The tail performance of the halfspace median is very appealing and also robust against underlying population distributions, while the tail performance of the sample mean is very sensitive to underlying population distributions. These findings provide new insights into the notions of the halfspace depth and breakdown point and identify the important role of tail behavior as a quantitative measure of robustness in the multivariate location setting.     

Bias correction of cross-validation criterion based on Kullback-Leibler information under a general condition

This paper deals with the bias correction of the cross-validation (CV) criterion to estimate the predictive Kullback-Leibler information. A bias-corrected CV criterion is proposed by replacing the ordinary maximum likelihood estimator with the maximizer of the adjusted log-likelihood function. The adjustment is just slight and simple, but the improvement of the bias is remarkable. The bias of the ordinary CV criterion is O(n^-1), but that of the bias-corrected CV criterion is O(n^-2). We verify that our criterion has smaller bias than the AIC, TIC, EIC and the ordinary CV criterion by numerical experiments.     

Multivariate analysis of variance with fewer observations than the dimension

In this article, we consider the problem of testing a linear hypothesis in a multivariate linear regression model which includes the case of testing the equality of mean vectors of several multivariate normal populations with common covariance matrix Σ, the so-called multivariate analysis of variance or MANOVA problem. However, we have fewer observations than the dimension of the random vectors. Two tests are proposed and their asymptotic distributions under the hypothesis as well as under the alternatives are given under some mild conditions. A theoretical comparison of these powers is made.     

Asymptotic robustness of standard errors in multilevel structural equation models

Data in social and behavioral sciences are often hierarchically organized. Multilevel statistical methodology was developed to analyze such data. Most of the procedures for analyzing multilevel data are derived from maximum likelihood based on the normal distribution assumption. Standard errors for parameter estimates in these procedures are obtained from the corresponding information matrix. Because practical data typically contain heterogeneous marginal skewnesses and kurtoses, this paper studies how nonnormally distributed data affect the standard errors of parameter estimates in a two-level structural equation model. Specifically, we study how skewness and kurtosis in one level affect standard errors of parameter estimates within its level and outside its level. We also show that, parallel to asymptotic robustness theory in conventional factor analysis, conditions exist for asymptotic robustness of standard errors in a multilevel factor analysis model.     

Progressive Type II censored order statistics for multivariate observations

For a sequence of independent and identically distributed random vectors , i=1,2,…,n, we consider the conditional ordering of these random vectors with respect to the magnitudes of , where N is a p-variate continuous function defined on the support set of X₁ and satisfying certain regularity conditions. We also consider the Progressive Type II right censoring for multivariate observations using conditional ordering. The need for the conditional ordering of random vectors exists for example, in reliability analysis when a system has n independent components each consisting of p arbitrarily dependent and parallel connected elements. Let the vector of life lengths for the ith component of the system be , where denotes the life length of the jth element of the ith component. Then the first failure in the system occurs at time , and for this case . In this paper we introduce the conditionally ordered and Progressive Type II right-censored conditionally ordered statistics for multivariate observations and to study their distributional properties.     

On Stein’s lemma, dependent covariates and functional monotonicity in multi-dimensional modeling

Tracking the correct directions of monotonicity in multi-dimensional modeling plays an important role in interpreting functional associations. In the presence of multiple predictors, we provide empirical evidence that the observed monotone directions via parametric, nonparametric or semiparametric fit of commonly used multi-dimensional models may entirely violate the actual directions of monotonicity. This breakdown is caused primarily by the dependence structure of covariates, with negligible influence from the bias of function estimation. To examine the linkage between the dependent covariates and monotone directions, we first generalize Stein’s Lemma for random variables which are mutually independent Gaussian to two important cases: dependent Gaussian, and independent non-Gaussian. We show that in both two cases, there is an explicit one-to-one correspondence between the monotone directions of a multi-dimensional function and the signs of a deterministic surrogate vector. Moreover, we demonstrate that the second case can be extended to accommodate a class of dependent covariates. This generalization further enables us to develop a de-correlation transform for arbitrarily dependent covariates. The transformed covariates preserve modeling interpretability with little loss in modeling efficiency. The simplicity and effectiveness of the proposed method are illustrated via simulation studies and real data application.     

Existence conditions for the uniformly minimum risk unbiased estimators in a class of linear models

This paper studies the existence of the uniformly minimum risk unbiased (UMRU) estimators of parameters in a class of linear models with an error vector having multivariate normal distribution or t-distribution, which include the growth curve model, the extended growth curve model, the seemingly unrelated regression equations model, the variance components model, and so on. The necessary and sufficient existence conditions are established for UMRU estimators of the estimable linear functions of regression coefficients under convex losses and matrix losses, respectively. Under the (extended) growth curve model and the seemingly unrelated regression equations model with normality assumption, the conclusions given in the literature can be derived by applying the general results in this paper. For the variance components model, the necessary and sufficient existence conditions are reduced as terse forms.     

Robust tests based on dual divergence estimators and saddlepoint approximations

This paper is devoted to robust hypothesis testing based on saddlepoint approximations in the framework of general parametric models. As is known, two main problems can arise when using classical tests. First, the models are approximations of reality and slight deviations from them can lead to unreliable results when using classical tests based on these models. Then, even if a model is correctly chosen, the classical tests are based on first order asymptotic theory. This can lead to inaccurate p-values when the sample size is moderate or small. To overcome these problems, robust tests based on dual divergence estimators and saddlepoint approximations, with good performances in small samples, are proposed.     

Limit theorems for permutations of empirical processes with applications to change point analysis

Theorems of approximation of Gaussian processes for the sequential empirical process of the permutations of independent random variables are established. The results are applied to simulate critical values for the functionals of sequential empirical processes used in change point analysis. The proofs are based on the properties of rank statistics and negatively associated random variables.