线性混合模型协方差阵的谱分解的一种新方法及其应用

对平衡线性混合模型, 随机效应的设计阵具有一定结构.定义了一种新的矩阵序, 借助于这种新序, 提出了协方差阵谱分解的一种新方法.与现有的两种方法相比较, 新方法的突出的特点是能够给出协方差阵不同特征值的精确个数, 以及谱分解中不同特征值对应的投影阵与随机效应的设计阵之间的关系. 基于新的谱分解结果,(1) 证明了平衡随机模型的方差分析估计为最小方差无偏估计; (2) 证明了在一定条件下, 一般平衡线性混合模型的方差分析估计也具有最小方差无偏性; (3) 给出了一般混合模型的极大似然方程显示解存在的一个较易验证的判定定理, 并给出了显示解存在时解的一般形式; (4) 清晰地显示了谱分解估计的构造原理, 并找到了谱分解估计与方差分析估计相等的充要条件.     

方差分量的广义谱分解估计

对于随机效应部分为一般平衡多向分类的线性混合模型,将王松桂(2002)提出的一种称之为谱分解估计的参数估计新方法推广到随机效应设计阵为任意矩阵的含两个方差分量的线性混合模型,给出了方差分量的广义谱分解估计方法,并证明了所得估计的一些统计性质。另外,还就广义谱分解估计类中某些特殊估计和对应的方差分析估计进行了比较,得到了它们相等的充分必要条件。     

方差分量模型协方差阵的谱分解

本文对平衡方差分量模型, 给出了其协方差阵的新的谱分解算法. 该方法的特点是计算简单, 易于理解, 无须复杂的数学知识. 且能够明确显示协方差阵的不同特征值的个数, 以及谱分解中不同特征值所对应的投影阵的显式表示. 基于新方法我们进一步研究了平衡方差分量模型的一些相关性质.本文还研究了一般方差分量模型, 我们首先定义了一般方差分量模型协方差阵的简单谱分解,给出了一般方差分量模型可以进行简单谱分解的充要条件, 并研究了协方差阵简单谱分解的一些性质. 对于协方差阵可以进行简单谱分解的方差分量模型, 本文研究了简单谱分解在其统计推断中的应用.     

随机变量二次型的协方差在混合效应模型中的应用

本文提出方差分量ANOVA估计的一种改进方法, 证明了对于一般的方差分量模型, 只要方差分量的ANOVA估计存在就可以通过此方法给出其改进形式, 并且在均方误差意义下优于ANOVA估计. 特别地, 对于单向分类随机效应模型, Kelly和Mathew^[1]对ANOVA估计的改进就是我们提出的改进方法的特殊形式, 这也给出了此类改进估计在均方误差意义下优于ANOVA估计的另一种合理的解释. 同时, 本文又将此思想应用到对谱分解估计的改进上. 本文应用协方差的简单性质证明了对带有一个随机效应的方差分量模型, 当随机效应的协方差阵只有一个非零特征值时, 随机效应方差分量谱分解估计在均方误差意义下总是优于ANOVA估计. 本文最后将第三节的结论推广到广义谱分解估计下, 同时给出广义谱分解估计待定系数的一个合理的取值.     

多元秩-序模型MLE的存在性

本文对多元秩 序模型极大似然估计的存在性进行了研究,在对模型协方差阵Ω的一些约束下,文中给出了其参数极大似然估计存在的一些充分必要条件.     

平衡线性混合模型方差分量几种估计的优良性

对于平衡线性混合模型,本文提出了一组易验证的条件,在此条件下,方差分量的谱分解估计、方差分析估计和最小范数二次无偏估计都相等且为一致最小方差无偏估计.同时证明了在此条件下,似然方程和限制似然方程都有显式解,还给出了许多满足这组条件的平衡线性混合模型的例子.     

平衡数据结构下ANOVA 估计和SD 估计的比较

在线性混合效应模型下, 方差分析(ANOVA) 估计和谱分解(SD) 估计对构造精确检验和广义P-值枢轴量起着非常重要的作用. 尽管这两估计分别基于不同的方法, 但它们共享许多类似的优点, 如无偏性和有精确的表达式等. 本文借助于已得到的协方差阵的谱分解结果, 揭示了平衡数据一般线性混合效应模型下ANOVA 估计与SD 估计的关系, 并分别针对协方差阵两种结构: 套结构和多项分类随机效应结构, 给出了ANOVA 估计与SD 估计等价的充分必要条件.     

混合效应模型下ANOVA估计和SD估计相等的充分条件

混合效应模型是统计模型中非常重要的一类模型,广泛地应用到许多领域.本文比较了该模型下方差分量的两种估计:方差分析(ANOVA)估计和谱分解(SD)估计,借助吴密霞和王松桂[A new method of spectral decomposition of covariance matrix in mixed effects models and its applications,Sci.China,Ser.A,2005,48:1451-1464]协方差矩阵的谱分解结果,给出了ANOVA估计和SD估计相等的两个充分条件及其相应的统计性质,并将以上的结果应用于圆形部件数据模型和混合方差分析模型.     

一般线性混合模型中的最佳线性无偏估计和谱分解估计

该文在一般线性混合模型中, 研究了固定和随机效应线性组合的估计问题.对观测向量的协方差阵可以为奇异矩阵情形下,导出了该组合的最佳线性无偏估计,并证明了它的唯一性.在一般线性混合模型的特例, 三个小域模型下, 得到了小域均值u_i 和方差分量的谱分解估计. 进而, 获得了基于谱分解估计的两步估计均方误差的二阶逼近.     

有限混合Laplace分布回归模型局部估计的EM算法（英文）

本文研究了一类有限混合Laplace分布回归模型的局部极大似然估计问题.利用核回归方法和最大化局部加权似然函数的EM算法,获得了参数函数的局部极大似然估计量,并讨论了它们的渐近偏差,渐近方差和渐近正态性.推广了有限混合回归模型下局部非参数估计的结果.     

A new method of spectral decomposition of covariance matrix in mixed effects models and its applications

For the mixed effects models with balanced data, a new ordering of design matrices of random effects is defined, and then a simple formula of the spectral decomposition of covariance matrix is obtained. To compare with the two methods in literature, the decomposition can not only give the actual number of all distinct eigenvalues and their expression, but also show clearly the relationship between the design matrices of random effects and the decomposition. These results can be applied to the problems for testifying the analysis of the variance estimate being a minimum variance unbiased under all random effects models and some mixed effects models with balanced data, for finding the explicit solution of maximum likelihood equations for the general mixed effects model and for showing the relationship between the spectral decomposition estimate and the analysis of variance estimate.     

Estimators of the regression parameters of the zeta distribution

The zeta distribution with regression parameters has been rarely used in statistics because of the difficulty of estimating the parameters by traditional maximum likelihood. We propose an alternative method for estimating the parameters based on an iteratively reweighted least-squares algorithm. The quadratic distance estimator (QDE) obtained is consistent, asymptotically unbiased and normally distributed; the estimate can also serve as the initial value required by an algorithm to maximize the likelihood function. We illustrate the method with a numerical example from the insurance literature; we compare the values of the estimates obtained by the quadratic distance and maximum likelihood methods and their approximate variance–covariance matrix. Finally, we calculate the bias, variance and the asymptotic efficiency of the QDE compared to the maximum likelihood estimator (MLE) for some values of the parameters.     

The use of generalized inverses in restricted maximum likelihood

The calculus of generalized inverses and related concepts in matrix algebra is applied to the general restricted maximum likelihood problem. Some new results on g-inverses, Kronecker products, and matrix differentials are presented. For the restricted maximum likelihood problem we obtain generalizations of the well-known results of Aitchison and Silvey [1]. We use the approach recently developed by Heijmans and Magnus [13, 14] to allow for non-i.i.d. observations. A nonlinear seemingly unrelated regressions model with possibly singular covariance matrix and linear restrictions (NLSURSR) is analyzed, and the linear expenditure system (LES) is discussed as a special case.     

Computation of reference Bayesian inference for variance components in longitudinal studies

Generalized linear mixed models (GLMMs) have been applied widely in the analysis of longitudinal data. This model confers two important advantages, namely, the flexibility to include random effects and the ability to make inference about complex covariances. In practice, however, the inference of variance components can be a difficult task due to the complexity of the model itself and the dimensionality of the covariance matrix of random effects. Here we first discuss for GLMMs the relation between Bayesian posterior estimates and penalized quasi-likelihood (PQL) estimates, based on the generalization of Harville’s result for general linear models. Next, we perform fully Bayesian analyses for the random covariance matrix using three different reference priors, two with Jeffreys’ priors derived from approximate likelihoods and one with the approximate uniform shrinkage prior. Computations are carried out via the combination of asymptotic approximations and Markov chain Monte Carlo methods. Under the criterion of the squared Euclidean norm, we compare the performances of Bayesian estimates of variance components with that of PQL estimates when the responses are non-normal, and with that of the restricted maximum likelihood (REML) estimates when data are assumed normal. Three applications and simulations of binary, normal, and count responses with multiple random effects and of small sample sizes are illustrated. The analyses examine the differences in estimation performance when the covariance structure is complex, and demonstrate the equivalence between PQL and the posterior modes when the former can be derived. The results also show that the Bayesian approach, particularly under the approximate Jeffreys’ priors, outperforms other procedures.     

An Inversion-Free Estimating Equations Approach for Gaussian Process Models

One of the scalability bottlenecks for the large-scale usage of Gaussian processes is the computation of the maximum likelihood estimates of the parameters of the covariance matrix. The classical approach requires a Cholesky factorization of the dense covariance matrix for each optimization iteration. In this work, we present an estimating equations approach for the parameters of zero-mean Gaussian processes. The distinguishing feature of this approach is that no linear system needs to be solved with the covariance matrix. Our approach requires solving an optimization problem for which the main computational expense for the calculation of its objective and gradient is the evaluation of traces of products of the covariance matrix with itself and with its derivatives. For many problems, this is an O(nlog?n) effort, and it is always no larger than O(n²). We prove that when the covariance matrix has a bounded condition number, our approach has the same convergence rate as does maximum likelihood in that the Godambe information matrix of the resulting estimator is at least as large as a fixed fraction of the Fisher information matrix. We demonstrate the effectiveness of the proposed approach on two synthetic examples, one of which involves more than 1 million data points.     

Consistent nonnegative estimates of variance components

In this paper, the estimation of variance components in the linear mixed model with two random effects is investigated. The class of combination estimates based on the quadratic invariant statistics and consistent nonnegative estimates are obtained. Furthermore, it is shown that the consistent nonnegative estimate dominates ANOVA estimate under some conditions.     

Relative efficiencies of estimates using patterned covariances or correlations in the multivariate normal estimation problem

Summary The relative efficiency of maximum likelihood estimates is studied when taking advantage of underlying linear patterns in the covariances or correlations when estimating covariance matrices. We compare the variances of estimates of the covariance matrix obtained under two nested patterns with the assumption that the more restricted pattern is the true state. Formulas for the asymptotic variances are given which are exact for linear covariance patterns when explicit maximum likelihood estimates exist. Several specific examples are given using complete symmetry, circular symmetry and general covariance patterns as well as an example involving a covariance matrix with a linear pattern in the correlations.     

Automatic model selection for partially linear models

We propose and study a unified procedure for variable selection in partially linear models. A new type of double-penalized least squares is formulated, using the smoothing spline to estimate the nonparametric part and applying a shrinkage penalty on parametric components to achieve model parsimony. Theoretically we show that, with proper choices of the smoothing and regularization parameters, the proposed procedure can be as efficient as the oracle estimator [J. Fan, R. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of American Statistical Association 96 (2001) 1348–1360]. We also study the asymptotic properties of the estimator when the number of parametric effects diverges with the sample size. Frequentist and Bayesian estimates of the covariance and confidence intervals are derived for the estimators. One great advantage of this procedure is its linear mixed model (LMM) representation, which greatly facilitates its implementation by using standard statistical software. Furthermore, the LMM framework enables one to treat the smoothing parameter as a variance component and hence conveniently estimate it together with other regression coefficients. Extensive numerical studies are conducted to demonstrate the effective performance of the proposed procedure.     

A Note on Random Effects Growth Curve Models

The robustness of regression coefficient estimator is a hot topic in regression analysis all the while. Since the response observations are not independent, it is extraordinarily difficult to study this problem for random effects growth curve models, especially when the design matrix is non-full of rank. The paper not only gives the necessary and sufficient conditions under which the generalized least square estimate is identical to the the best linear unbiased estimate when error covariance matrix is an arbitrary positive definite matrix, but also obtains a concise condition under which the generalized least square estimate is identical to the maximum likelihood estimate when the design matrix is full or non-full of rank respectively. In addition, by using of the obtained results, we get some corollaries for the the generalized least square estimate be equal to the maximum likelihood estimate under several common error covariance matrix assumptions. Illustrative examples for the case that the design matrix is full or non-full of rank are also given.     

Conditional and unconditional methods for selecting variables in linear mixed models

In the problem of selecting the explanatory variables in the linear mixed model, we address the derivation of the (unconditional or marginal) Akaike information criterion (AIC) and the conditional AIC (cAIC). The covariance matrices of the random effects and the error terms include unknown parameters like variance components, and the selection procedures proposed in the literature are limited to the cases where the parameters are known or partly unknown. In this paper, AIC and cAIC are extended to the situation where the parameters are completely unknown and they are estimated by the general consistent estimators including the maximum likelihood (ML), the restricted maximum likelihood (REML) and other unbiased estimators. We derive, related to AIC and cAIC, the marginal and the conditional prediction error criteria which select superior models in light of minimizing the prediction errors relative to quadratic loss functions. Finally, numerical performances of the proposed selection procedures are investigated through simulation studies.