Estimation of Wishart mean matrices under simple tree ordering

For Wishart density functions, we study the risk dominance problems of the restricted maximum likelihood estimators of mean matrices with respect to the Kullback-Leibler loss function over restricted parameter space under the simple tree ordering set. The results are directly applied to the estimation of covariance matrices for the completely balanced multivariate multi-way random effects models without interactions.     

A new method of spectral decomposition of covariance matrix in mixed effects models and its applications

For the mixed effects models with balanced data, a new ordering of design matrices of random effects is defined, and then a simple formula of the spectral decomposition of covariance matrix is obtained. To compare with the two methods in literature, the decomposition can not only give the actual number of all distinct eigenvalues and their expression, but also show clearly the relationship between the design matrices of random effects and the decomposition. These results can be applied to the problems for testifying the analysis of the variance estimate being a minimum variance unbiased under all random effects models and some mixed effects models with balanced data, for finding the explicit solution of maximum likelihood equations for the general mixed effects model and for showing the relationship between the spectral decomposition estimate and the analysis of variance estimate.     

Conditional and unconditional methods for selecting variables in linear mixed models

In the problem of selecting the explanatory variables in the linear mixed model, we address the derivation of the (unconditional or marginal) Akaike information criterion (AIC) and the conditional AIC (cAIC). The covariance matrices of the random effects and the error terms include unknown parameters like variance components, and the selection procedures proposed in the literature are limited to the cases where the parameters are known or partly unknown. In this paper, AIC and cAIC are extended to the situation where the parameters are completely unknown and they are estimated by the general consistent estimators including the maximum likelihood (ML), the restricted maximum likelihood (REML) and other unbiased estimators. We derive, related to AIC and cAIC, the marginal and the conditional prediction error criteria which select superior models in light of minimizing the prediction errors relative to quadratic loss functions. Finally, numerical performances of the proposed selection procedures are investigated through simulation studies.     

Estimation of means of multivariate normal populations with order restriction

Multivariate isotonic regression theory plays a key role in the field of statistical inference under order restriction for vector valued parameters. Two cases of estimating multivariate normal means under order restricted set are considered. One case is that covariance matrices are known, the other one is that covariance matrices are unknown but are restricted by partial order. This paper shows that when covariance matrices are known, the estimator given by this paper always dominates unrestricted maximum likelihood estimator uniformly, and when covariance matrices are unknown, the plug-in estimator dominates unrestricted maximum likelihood estimator under the order restricted set of covariance matrices. The isotonic regression estimators in this paper are the generalizations of plug-in estimators in unitary case.     

Maximum likelihood estimation of Wishart mean matrices under Löwner order restrictions

For Wishart density functions, there remains a long-time question unsolved. That is whether there exists the closed-form MLEs of mean matrices over the partially Löwner ordering sets. In this note, we provide an affirmative answer by demonstrating a unified procedure on exactly how the closed-form MLEs are obtained for the simple ordering case. Under the Kullback-Leibler loss function, a property of obtained MLEs is further studied. Some applications of the obtained closed-form MLEs, including the comparison between our ML estimates and Calvin and Dykstra's [Maximum likelihood estimation of a set of covariance matrices under Löwner order restrictions with applications to balanced multivariate variance components models, Ann. Statist. 19 (1991) 850-869.] which obtained by iterative algorithm, are also made.     

平衡数据结构下ANOVA 估计和SD 估计的比较

在线性混合效应模型下, 方差分析(ANOVA) 估计和谱分解(SD) 估计对构造精确检验和广义P-值枢轴量起着非常重要的作用. 尽管这两估计分别基于不同的方法, 但它们共享许多类似的优点, 如无偏性和有精确的表达式等. 本文借助于已得到的协方差阵的谱分解结果, 揭示了平衡数据一般线性混合效应模型下ANOVA 估计与SD 估计的关系, 并分别针对协方差阵两种结构: 套结构和多项分类随机效应结构, 给出了ANOVA 估计与SD 估计等价的充分必要条件.     

The maximum likelihood estimators in a multivariate normal distribution with AR(1) covariance structure for monotone data

The maximum likelihood estimators are uniquely obtained in a multivariate normal distribution with AR(1) covariance structure for monotone data. The maximum likelihood estimator of mean is unbiased.     

Tapered Covariance: Bayesian Estimation and Asymptotics

The method of maximum tapered likelihood has been proposed as a way to quickly estimate covariance parameters for stationary Gaussian random fields. We show that under a useful asymptotic regime, maximum tapered likelihood estimators are consistent and asymptotically normal for covariance models in common use. We then formalize the notion of tapered quasi-Bayesian estimators and show that they too are consistent and asymptotically normal. We also present asymptotic confidence intervals for both types of estimators and show via simulation that they accurately reflect sampling variability, even at modest sample sizes. Proofs, an example, and detailed derivations are provided in the supplementary materials, available online.     

On the quadratic estimation of covariance matrices in multivariate linear models

This paper investigates the estimation of covariance matrices in multivariate mixed models. Some sufficient conditions are derived for a multivariate quadratic form and a linear combination of multivariate quadratic forms to be the BQUE (quadratic unbiased and severally minimum varianced) estimators of its expectations.     

Estimation of the entropy of a multivariate normal distribution

Motivated by problems in molecular biosciences wherein the evaluation of entropy of a molecular system is important for understanding its thermodynamic properties, we consider the efficient estimation of entropy of a multivariate normal distribution having unknown mean vector and covariance matrix. Based on a random sample, we discuss the problem of estimating the entropy under the quadratic loss function. The best affine equivariant estimator is obtained and, interestingly, it also turns out to be an unbiased estimator and a generalized Bayes estimator. It is established that the best affine equivariant estimator is admissible in the class of estimators that depend on the determinant of the sample covariance matrix alone. The risk improvements of the best affine equivariant estimator over the maximum likelihood estimator (an estimator commonly used in molecular sciences) are obtained numerically and are found to be substantial in higher dimensions, which is commonly the case for atomic coordinates in macromolecules such as proteins. We further establish that even the best affine equivariant estimator is inadmissible and obtain Stein-type and Brewster–Zidek-type estimators dominating it. The Brewster–Zidek-type estimator is shown to be generalized Bayes.     

Marginal permutation invariant covariance matrices with applications to linear models

The goal of the present paper is to perform a comprehensive study of the covariance structures in balanced linear models containing random factors which are invariant with respect to marginal permutations of the random factors. We shall focus on model formulation and interpretation rather than the estimation of parameters. It is proven that permutation invariance implies a specific structure for the covariance matrices. Useful results are obtained for the spectra of permutation invariant covariance matrices. In particular, the reparameterization of random effects, i.e., imposing certain constraints, will be considered. There are many possibilities to choose reparameterization constraints in a linear model, however not every reparameterization keeps permutation invariance. The question is if there are natural restrictions on the random effects in a given model, i.e., such reparameterizations which are defined by the covariance structure of the corresponding factor. Examining relationships between the reparameterization conditions applied to the random factors of the models and the spectrum of the corresponding covariance matrices when permutation invariance is assumed, restrictions on the spectrum of the covariance matrix are obtained which lead to “sum-to-zero” reparameterization of the corresponding factor.     

On estimation in some reduced rank extended growth curve models

The general multivariate analysis of variance model has been extensively studied in the statistical literature and successfully applied in many different fields for analyzing longitudinal data. In this article, we consider the extension of this model having two sets of regressors constituting a growth curve portion and a multivariate analysis of variance portion, respectively. Nowadays, the data collected in empirical studies have relatively complex structures though often demanding a parsimonious modeling. This can be achieved for example through imposing rank constraints on the regression coefficient matrices. The reduced rank regression structure also provides a theoretical interpretation in terms of latent variables. We derive likelihood based estimators for the mean parameters and covariance matrix in this type of models. A numerical example is provided to illustrate the obtained results.     

有缺失数据的条件独立正态母体中参数的最优同变估计

在缺失数据机制是可忽略的假设下,导出了有单调缺失数据的条件独立正态模型中协方差阵和精度阵的Cholesky分解的最大似然估计和无偏估计.通过引入一类特殊的变换群并在更广义的损失下,获得了其最优同变估计.这表明最大似然估计和无偏估计是非容许的.最后,通过数值模拟验证了相关结果的有效性.     

Estimation of covariance matrices in fixed and mixed effects linear models

The estimation of the covariance matrix or the multivariate components of variance is considered in the multivariate linear regression models with effects being fixed or random. In this paper, we propose a new method to show that usual unbiased estimators are improved on by the truncated estimators. The method is based on the Stein–Haff identity, namely the integration by parts in the Wishart distribution, and it allows us to handle the general types of scale-equivariant estimators as well as the general fixed or mixed effects linear models.     

Generalized variance estimators in the multivariate gamma models

It is already known that the uniformly minimum variance unbiased (UMVU) estimator of the generalized variance always exists for any natural exponential family. However, in practice, this estimator is often difficult to obtain. This paper provides explicit forms of the UMVU estimators for the bivariate and symmetric multivariate gamma models, which are diagonal quadratic exponential families. For the non-independent multivariate gamma models, it is shown that the UMVU and the maximum likelihood estimators are not proportional.      

CRPS M-estimation for max-stable models

Max-stable processes provide canonical models for the dependence of multivariate extremes. Inference with such models has been challenging due to the lack of tractable likelihoods which has motivated use of composite likelihood methods Padoan et al. (J. Amer. Stat. Assoc. 105(489):263–277, 2010). In contrast, the finite dimensional cumulative distribution functions (CDFs) appear natural to work with, and are readily available or can be approximated well. Motivated by this fact, in this work we develop an M-estimation framework for max-stable models based on the continuous ranked probability score (CRPS) of multivariate CDFs. We start by establishing conditions for the consistency and asymptotic normality of the CRPS-based estimators in a general context. We then implement them in the max-stable setting and provide readily computable expressions for their asymptotic covariance matrices. The resulting point and asymptotic confidence interval estimates are illustrated over popular simulated models. They enjoy accurate coverages and offer an alternative to composite likelihood based methods.     

半参数线性混合效应模型的联合变量选择

多数基于线性混合效应模型的变量选择方法分阶段对固定效应和随机效应进行选择,方法繁琐、易产生模型偏差,且大部分非参数和半参数的线性混合效应模型只涉及非参数部分的光滑度或者固定效应的选择,并未涉及非参变量或随机效应的选择。本文用B样条函数逼近非参数函数部分,从而把半参数线性混合效应模型转化为带逼近误差的线性混合效应模型。对随机效应的协方差矩阵采用改进的乔里斯基分解并重新参数化线性混合效应模型,接着对该模型的极大似然函数施加集群ALASSO惩罚和ALASSO惩罚两类惩罚,该法能实现非参数变量、固定效应和随机效应的联合变量选择,基于该法得出的估计量也满足相合性、稀疏性和Oracle性质。文章最后做了个数值模拟,模拟结果表明,本文提出的估计方法在变量选择的准确性、参数估计的精度两个方面均表现较好。     

异方差的线性混合模型的参数估计

本文研究既含有固定效应又含有随机效应的线性混合模型,在随机效应的方差不同即异方差情况下,即考虑方差受外界因素的影响,如温度、湿度等,我们称之为协变量,在有协变量情况下对方差建立对数线性模型,运用最大似然估计讨论了固定效应的估计和随机效应的预测,并且用约束最大似然(REML)方法研究对数线性模型中参数和随机误差中参数(离差参数)的估计,并讨论估计量的性质及离差参数估计量的渐近正态性。     

增长曲线模型中一致最小风险无偏估计的存在性

考虑协方差阵任意，或具有均匀协方差结构，或具有序列协方差结构的正态增长曲线模型本文将文[19]在设计矩阵满秩，且仅估计回归系数矩阵的情形获得的结果推广到设计矩阵不必列满秩，且同时估计回归系数矩阵的线性可估函数和协方差阵（或有关参数）的情形；在凸损失函数类和矩阵损失函数下，给出存在一致最小风险无偏估计的充分必要条件．     

A general class of scale-shape mixtures of skew-normal distributions: properties and estimation

This paper introduces the scale-shape mixtures of skew-normal (SSMSN) distributions which provide alternative candidates for modeling asymmetric data in a wide variety of settings. We obtain the moments and study some characterizations of the SSMSN distributions. Instead of resorting to numerical optimization procedures, two variants of EM algorithms are developed for carrying out maximum likelihood estimation. Our algorithms are analytically simple because closed-form expressions of conditional expectations in the E-step as well as the updating estimators in the M-step can be explicitly obtained. The observed information matrix is derived for approximating the asymptotic covariance matrix of parameter estimates. A simulation study is conducted to examine the finite sample properties of ML estimators. The utility of the proposed methodology is illustrated by analyzing a real example.