Estimation of means of multivariate normal populations with order restriction

Multivariate isotonic regression theory plays a key role in the field of statistical inference under order restriction for vector valued parameters. Two cases of estimating multivariate normal means under order restricted set are considered. One case is that covariance matrices are known, the other one is that covariance matrices are unknown but are restricted by partial order. This paper shows that when covariance matrices are known, the estimator given by this paper always dominates unrestricted maximum likelihood estimator uniformly, and when covariance matrices are unknown, the plug-in estimator dominates unrestricted maximum likelihood estimator under the order restricted set of covariance matrices. The isotonic regression estimators in this paper are the generalizations of plug-in estimators in unitary case.     

Selection of variables in two-group discriminant analysis by error rate and Akaike's information criteria

This paper deals with two criteria for selection of variables for the discriminant analysis in the case of two multivariate normal populations with different means and a common covariance matrix. One is based on the estimated error rate of misclassification. The other uses Akaike's information criterion. The asymptotic distributions and error rate risks of the criteria are obtained. The result will prove that the two criteria are asymptotically equivalent in the sense of their asymptotic distributions and error rate risks being identical.     

误差服从多元t分布的一类线性模型下参数估计的若干注记

研究一类线性模型下参数估计的若干问题．这类模型包含了多个因变量线性模型、增长曲线模型、扩充的增长曲线模型、似乎不相关回归方程组、方差分量模型等常用模型．在这类线性模型下,证明了当误差服从多元t分布时与误差服从多元正态分布时,具有相同的完全统计量和无偏估计,且在后一种情况下的充分统计量必为前一种情况下的充分统计量．对于带有多种协方差结构的前述几种模型,把在误差服从多元正态分布下,相应的协方差阵及有关参数的一致最小风险无偏(UMRU)估计存在性的结论推广到了相应的误差服从多元t分布情形．此外,对于误差服从多元t分布的这类统一的线性模型,给出了回归系数的线性可估函数的无偏估计的协方差阵的C-R下界．     

The efficiency of logistic regression compared to normal discriminant analysis under class-conditional classification noise

In many real world classification problems, class-conditional classification noise (CCC-Noise) frequently deteriorates the performance of a classifier that is naively built by ignoring it. In this paper, we investigate the impact of CCC-Noise on the quality of a popular generative classifier, normal discriminant analysis (NDA), and its corresponding discriminative classifier, logistic regression (LR). We consider the problem of two multivariate normal populations having a common covariance matrix. We compare the asymptotic distribution of the misclassification error rate of these two classifiers under CCC-Noise. We show that when the noise level is low, the asymptotic error rates of both procedures are only slightly affected. We also show that LR is less deteriorated by CCC-Noise compared to NDA. Under CCC-Noise contexts, the Mahalanobis distance between the populations plays a vital role in determining the relative performance of these two procedures. In particular, when this distance is small, LR tends to be more tolerable to CCC-Noise compared to NDA.     

Optimal training sample allocation and asymptotic expansions for error rates in discriminant analysis

The sample-based rule obtained from Bayes classification rule by replacing the unknown parameters by ML estimates from a stratified training sample is used for the classification of a random observationX into one ofL populations. The asymptotic expansions in terms of the inverses of the training sample sizes for cross-validation, apparent and plug-in error rates are found. These are used to compare estimation methods of the error rate for a wide range of regular distributions as probability models for considered populations. The optimal training sample allocation minimizing the asymptotic expected error regret is found in the cases of widely applicable, positively skewed distributions (Rayleigh and Maxwell distributions). These probability models for populations are often met in ecology and biology. The results indicate that equal training sample sizes for each populations sometimes are not optimal, even when prior probabilities of populations are equal.     

Distribution of discriminant function in circular models

Summary Assuming that the covariance matrices are circular, we make an appropriate transformation which reduces the circular matrices to canonical forms. The discriminant function is given when the populations are multivariate normal with different circular matrices and its distribution is derived. An asymptotic expansion for the distribution is obtained when all the parameters are unknown. This research is partially supported by National Institutes of Health Grant No. GM 00034-12.     

The Covariance Adjusted Location Linear Discriminant Function for Classifying Data with Unequal Dispersion Matrices in Different Locations

Classification between two populations dealing with both continuous and binary variables is handled by splitting the problem into different locations. Given the location specified by the values of the binary variables, discrimination is performed using the continuous variables. The location probability model with homoscedastic across location conditional dispersion matrices is adopted for this problem. In this paper, we consider presence of continuous covariates with heterogeneous location conditional dispersion matrices. The continuous covariates have equal location specific mean in both populations. Conditional homoscedasticity fails when strong interaction between the continuous and binary variables is present. A plug-in covariance adjusted rule is constructed and its asymptotic distribution is derived. An asymptotic expansion for the overall error rate is given. The result is extended to include binary covariates.     

正态-逆Wishart先验下多元线性模型中经验Bayes估计的优良性

在正态-逆Wishart先验下研究了多元线性模型中参数的经验Bayes估计及其优良性问题.当先验分布中含有未知参数时,构造了回归系数矩阵和误差方差矩阵的经验Bayes估计,并在Bayes均方误差(简称BMSE)准则和Bayes均方误差阵(简称BMSEM)准则下,证明了经验Bayes估计优于最小二乘估计.最后,进行了Monte Carlo模拟研究,进一步验证了理论结果.     

Discrimination with jointly equicorrelated multi-level multivariate data

In this article we study a linear as well as a quadratic discriminant function for multi-level multivariate repeated measurement data under the assumption of multivariate normality. We assume that the m-variate observations have jointly equicorrelated covariance structure in addition to a Kronecker product structure on the mean vector. The new discriminant functions are very effective in discriminating individuals when the number of observations is very small. The proposed classification rules are demonstrated on a real data set. The error rates of the proposed classification rules are found to be much less than the error rates of the traditional classification rules, when in fact the traditional classification rules fail most of the time owing to the small sample sizes.     

Discriminant Analysis When a Block of Observations is Missing

We consider the problem of classifying a p× 1 observation into one of two multivariate normal populations when the training samples contain a block of missing observations. A new classification procedure is proposed which is a linear combination of two discriminant functions, one based on the complete samples and the other on the incomplete samples. The new discriminant function is easy to use. We consider the estimation of error rate of the linear combination classification procedure by using the leave-one-out estimation and bootstrap estimation. A Monte Carlo study is conducted to evaluate the error rate and the estimation of it. A numerical example is given tof illustrate its use.     

Rank estimation in reduced-rank regression

Reduced rank regression assumes that the coefficient matrix in a multivariate regression model is not of full rank. The unknown rank is traditionally estimated under the assumption of normal responses. We derive an asymptotic test for the rank that only requires the response vector have finite second moments. The test is extended to the nonconstant covariance case. Linear combinations of the components of the predictor vector that are estimated to be significant for modelling the responses are obtained.     

High Breakdown Estimation for Multiple Populations with Applications to Discriminant Analysis

We consider S-estimators of multivariate location and common dispersion matrix in multiple populations. Instead of averaging the robust estimates of the individual covariance matrices, as used by Todorov, Neykov and Neytchev (1990), the observations are pooled for estimating the common covariance more efficiently. Two such proposals are evaluated by a breakdown point analysis and Monte Carlo simulations. Their applications to the discriminant analysis are also considered.     

On using influence functions for testing multivariate normality

Changes in the joint distribution of influence functions for the mean vector and the covariance matrix are examined when the true probability distribution is contaminated. In particular, the formulas for influence functions of the first and second moments with respect to the above joint distribution are obtained and used to derive reasonable test statistics for multivariate normality. The formulas are extended by using the joint distribution of score functions for population parameters. An application of the extended formulas to the usual linear regression analysis leads to a measure of multivariate skewness which can be used to reduce the effect of non-normality of the response variable. Also, some relationship between the extended formulas and goodness-of-fit statistics is discussed and used to derive test statistics for multivariate normality.     

The multivariate least-trimmed squares estimator

In this paper we introduce the least-trimmed squares estimator for multivariate regression. We give three equivalent formulations of the estimator and obtain its breakdown point. A fast algorithm for its computation is proposed. We prove Fisher-consistency at the multivariate regression model with elliptically symmetric error distribution and derive the influence function. Simulations investigate the finite-sample efficiency and robustness of the estimator. To increase the efficiency of the estimator, we also consider a one-step reweighted estimator.     

Nonparametric estimation of multivariate density with direct and auxiliary data and application

We consider the problem of multivariate density estimation, using samples from the distribution of interest as well as auxiliary samples from a related distribution. We assume that the data from the target distribution and the related distribution may occur individually as well as in pairs. Using nonparametric maximum likelihood estimator of the joint distribution, we derive a kernel density estimator of the marginal density. We show theoretically, in a simple special case, that the implied estimator of the marginal density has smaller integrated mean squared error than that of a similar estimator obtained by ignoring dependence of the paired observations. We establish consistency of the marginal density estimator under suitable conditions. We demonstrate small sample superiority of the proposed estimator over the estimator that ignores dependence of the samples, through a simulation study with dependent and non-normal populations. The application of the density estimator in nonparametric classification is also discussed. It is shown that the misclassification probability of the resulting classifier is asymptotically equivalent to that of the Bayes classifier. We also include a data analytic illustration.     

Asymptotic distribution of the discriminant function

This paper describes discrimination among multivariate autoregressive processes by the Bayes method. The asymptotic distribution of the discriminant function and estimation of the probability of misclassification are investigated.     

Inference for robust canonical variate analysis

We consider the problem of optimally separating two multivariate populations. Robust linear discriminant rules can be obtained by replacing the empirical means and covariance in the classical discriminant rules by S or MM-estimates of location and scatter. We propose to use a fast and robust bootstrap method to obtain inference for such a robust discriminant analysis. This is useful since classical bootstrap methods may be unstable as well as extremely time-consuming when robust estimates such as S or MM-estimates are involved. In particular, fast and robust bootstrap can be used to investigate which variables contribute significantly to the canonical variate, and thus the discrimination of the classes. Through bootstrap, we can also examine the stability of the canonical variate. We illustrate the method on some real data examples.     

Estimation of the entropy of a multivariate normal distribution

Motivated by problems in molecular biosciences wherein the evaluation of entropy of a molecular system is important for understanding its thermodynamic properties, we consider the efficient estimation of entropy of a multivariate normal distribution having unknown mean vector and covariance matrix. Based on a random sample, we discuss the problem of estimating the entropy under the quadratic loss function. The best affine equivariant estimator is obtained and, interestingly, it also turns out to be an unbiased estimator and a generalized Bayes estimator. It is established that the best affine equivariant estimator is admissible in the class of estimators that depend on the determinant of the sample covariance matrix alone. The risk improvements of the best affine equivariant estimator over the maximum likelihood estimator (an estimator commonly used in molecular sciences) are obtained numerically and are found to be substantial in higher dimensions, which is commonly the case for atomic coordinates in macromolecules such as proteins. We further establish that even the best affine equivariant estimator is inadmissible and obtain Stein-type and Brewster–Zidek-type estimators dominating it. The Brewster–Zidek-type estimator is shown to be generalized Bayes.     

Bayesian shrinkage prediction for the regression problem

We consider Bayesian shrinkage predictions for the Normal regression problem under the frequentist Kullback-Leibler risk function.Firstly, we consider the multivariate Normal model with an unknown mean and a known covariance. While the unknown mean is fixed, the covariance of future samples can be different from that of training samples. We show that the Bayesian predictive distribution based on the uniform prior is dominated by that based on a class of priors if the prior distributions for the covariance and future covariance matrices are rotation invariant.Then, we consider a class of priors for the mean parameters depending on the future covariance matrix. With such a prior, we can construct a Bayesian predictive distribution dominating that based on the uniform prior.Lastly, applying this result to the prediction of response variables in the Normal linear regression model, we show that there exists a Bayesian predictive distribution dominating that based on the uniform prior. Minimaxity of these Bayesian predictions follows from these results.     

AN ANALYSIS OF AMULTIVARIATE TWO－WAY MODEL WITH INTERACTION AND NO REPLICATION

ＡＮＡＮＡＬＹＳＩＳＯＦＡＭＵＬＴＩＶＡＲＩＡＴＥＴＷＯ－ＷＡＹＭＯＤＥＬＷＩＴＨＩＮＴＥＲＡＣＴＩＯＮＡＮＤＮＯＲＥＰＬＩＣＡＴＩＯＮＧＵＯＤＡＷＥＩ（郭大伟）（ＤｅｐａｒｔｍｅｎｔｏｆＭａｔｈｅｍａｔｉｃｓ，ＡｎｈｕｉＮｏｒｍａｌＵｎｉｖｅｒｓｉ...