MM Algorithms for Variance Components Models

Variance components estimation and mixed model analysis are central themes in statistics with applications in numerous scientific disciplines. Despite the best efforts of generations of statisticians and numerical analysts, maximum likelihood estimation (MLE) and restricted MLE of variance component models remain numerically challenging. Building on the minorization–maximization (MM) principle, this article presents a novel iterative algorithm for variance components estimation. Our MM algorithm is trivial to implement and competitive on large data problems. The algorithm readily extends to more complicated problems such as linear mixed models, multivariate response models possibly with missing data, maximum a posteriori estimation, and penalized estimation. We establish the global convergence of the MM algorithm to a Karush–Kuhn–Tucker point and demonstrate, both numerically and theoretically, that it converges faster than the classical EM algorithm when the number of variance components is greater than two and all covariance matrices are positive definite. Supplementary materials for this article are available online.     

Joint modeling of failure time data with transformation model and longitudinal data when covariates are measured with errors

Semiparametric transformation models provide a class of flexible models for regression analysis of failure time data. Several authors have discussed them under different situations when covariates are timeindependent (Chen et al., 2002; Cheng et al., 1995; Fine et al., 1998). In this paper, we consider fitting these models to right-censored data when covariates are time-dependent longitudinal variables and, furthermore, may suffer measurement errors. For estimation, we investigate the maximum likelihood approach, and an EM algorithm is developed. Simulation results show that the proposed method is appropriate for practical application, and an illustrative example is provided.     

���ɾʧ�����»��ָ���ֲ��Ĺ���

The hybrid censoring scheme is a mixture of type-I and type-II censoring schemes. It is a popular censoring scheme in the literature of life data analysis. Mixed exponential distribution (MED) models is a class of favorable models in reliability statistics. Nevertheless, there is no much discussion to focus on parameters estimation for MED models with hybrid censored samples. We will address this problem in this paper. The EM (Expectation-Maximization) algorithm is employed to derive the closed form of the maximum likelihood estimators (MLEs). Finally, Monte Carlo simulations and a real-world data analysis are conducted to illustrate the proposed method.     

A New,Fast Algorithm to Find the Regions of Possible Support for Bivariate Interval-Censored Data

The estimation of the nonparametric maximum likelihood estimate (NPMLE) of the bivariate distribution function on interval-censored data is a recent topic of research. Among other things, it provides a basic tool for checking a parametric model for the bivariate failure times. As a first step in the estimation of the NPMLE for bivariate interval-censored data, the regions of possible support—that is, the rectangles with nonzero mass—are calculated. For this step a new, fast algorithm is introduced here and compared with two existing algorithms. The advantages of our algorithm will be illustrated on the emergence times of permanent teeth on data from the longitudinal Signal® Tandmobiel study.     

Estimation of parameters in latent class models using fuzzy clustering algorithms

A mixture approach to clustering is an important technique in cluster analysis. A mixture of multivariate multinomial distributions is usually used to analyze categorical data with latent class model. The parameter estimation is an important step for a mixture distribution. Described here are four approaches to estimating the parameters of a mixture of multivariate multinomial distributions. The first approach is an extended maximum likelihood (ML) method. The second approach is based on the well-known expectation maximization (EM) algorithm. The third approach is the classification maximum likelihood (CML) algorithm. In this paper, we propose a new approach using the so-called fuzzy class model and then create the fuzzy classification maximum likelihood (FCML) approach for categorical data. The accuracy, robustness and effectiveness of these four types of algorithms for estimating the parameters of multivariate binomial mixtures are compared using real empirical data and samples drawn from the multivariate binomial mixtures of two classes. The results show that the proposed FCML algorithm presents better accuracy, robustness and effectiveness. Overall, the FCML algorithm has the superiority over the ML, EM and CML algorithms. Thus, we recommend FCML as another good tool for estimating the parameters of mixture multivariate multinomial models.     

A Gaussian Mixed Model for Learning Discrete Bayesian Networks

In this paper, we address the problem of learning discrete Bayesian networks from noisy data. A graphical model based on a mixture of Gaussian distributions with categorical mixing structure coming from a discrete Bayesian network is considered. The network learning is formulated as a maximum likelihood estimation problem and performed by employing an EM algorithm. The proposed approach is relevant to a variety of statistical problems for which Bayesian network models are suitable—from simple regression analysis to learning gene/protein regulatory networks from microarray data.     

正态分布和瑞利分布混合情形下的参数估计及分类问题

学者往往用单一的分布模拟和拟合杂波,如正态分布、瑞利分布和威布尔分布等。然而在实际中,雷达杂波由多种类型的杂波组成,单一分布通常不能精确刻画雷达杂波规律,因此,应用混合分布模型对雷达杂波数据建模更准确。本文考虑用正态分布和瑞利分布的混合分布拟合杂波,并应用矩估计方法和基于EM算法的极大似然估计方法估计模型参数,最后,应用最大后验概率分类准则验证2种估计方法的分类准确率。通过数据模拟,得出极大似然估计的效果和分类准确率都要优于矩估计的估计效果和分类准确率。     

Composite likelihood and maximum likelihood methods for joint latent class modeling of disease prevalence and high-dimensional semicontinuous biomarker data

Joint latent class modeling of disease prevalence and high-dimensional semicontinuous biomarker data has been proposed to study the relationship between diseases and their related biomarkers. However, statistical inference of the joint latent class modeling approach has proved very challenging due to its computational complexity in seeking maximum likelihood estimates. In this article, we propose a series of composite likelihoods for maximum composite likelihood estimation, as well as an enhanced Monte Carlo expectation–maximization (MCEM) algorithm for maximum likelihood estimation, in the context of joint latent class models. Theoretically, the maximum composite likelihood estimates are consistent and asymptotically normal. Numerically, we have shown that, as compared to the MCEM algorithm that maximizes the full likelihood, not only the composite likelihood approach that is coupled with the quasi-Newton method can substantially reduce the computational complexity and duration, but it can simultaneously retain comparative estimation efficiency.     

Automatic Differentiation to Facilitate Maximum Likelihood Estimation in Nonlinear Random Effects Models

Maximum likelihood estimation in random effects models for non-Gaussian data is a computationally challenging task that currently receives much attention. This article shows that the estimation process can be facilitated by the use of automatic differentiation, which is a technique for exact numerical differentiation of functions represented as computer programs. Automatic differentiation is applied to an approximation of the likelihood function, obtained by using either Laplace's method of integration or importance sampling. The approach is applied to generalized linear mixed models. The computational speed is high compared to the Monte Carlo EM algorithm and the Monte Carlo Newton–Raphson method.     

偏t正态数据下混合线性联合位置与尺度模型的参数估计

偏t正态分布是分析尖峰,厚尾数据的重要统计工具之一.研究提出了偏t正态数据下混合线性联合位置与尺度模型,通过EM算法和Newton-Raphson方法研究了该模型参数的极大似然估计.并通过随机模拟试验验证了所提出方法的有效性.最后,结合实际数据验证了该模型和方法具有实用性和可行性.     

偏正态数据下众数回归模型的统计诊断

为了更好地拟合偏态数据,充分提取偏态数据的信息,针对偏正态数据建立了众数回归模型,并基于Pena距离统计量对众数回归模型进行统计断研究,得到了众数回归模型的Pena距离表达式以及高杠杆异常点的诊断方法.利用EM算法与梯度下降法给出了众数回归模型参数的极大似然估计,根据数据删除模型计算似然距离、Cook距离和Pena距离统计量,绘制诊断统计图.通过Monte Carlo模拟试验和实例分析比较,说明文章提出的方法行之有效,并在一定条件下Pena距离对异常点或强影响点的诊断优于似然距离和Cook距离.     

基于删失数据的指数威布尔分布最大似然估计的新算法

本文讨论了指数威布尔分布当观测数据是删失数据情形时参数的最大似然估计问题.因为删失数据是一种不完全数据,我们利用EM算法来计算参数的近似最大似然估计.由于EM算法计算的复杂性,计算效率也不理想.为了克服牛顿-拉普森算法和EM算法的局限性,我们提出了一种新的方法.这种方法联合了指数威布尔分布到指数分布的变换和等效寿命数据的技巧,比牛顿-拉普森算法和EM算法更具有操作性.数据模拟讨论了这一方法的可行性.为了演示本文的方法,我们还提供了一个真实寿命数据分析的例子.     

一类零膨胀半参数混合效应模型的惩罚似然估计

本文将半参数线性混合效应模型推广应用到一类具有零膨胀的纵向数据或集群数据的研究中,提出了一类新的半参数混合效应模型,然后利用广义交叉核实法选取光滑参数,通过最大惩罚似然函数方法与EM算法给出了模型参数部分与非参数部分的估计方法,最后,通过模拟和实例说明了本文方法的有效性.     

Piecewise Linear Approximations for Cure Rate Models and Associated Inferential Issues

Cure rate models offer a convenient way to model time-to-event data by allowing a proportion of individuals in the population to be completely cured so that they never face the event of interest (say, death). The most studied cure rate models can be defined through a competing cause scenario in which the random variables corresponding to the time-to-event for each competing causes are conditionally independent and identically distributed while the actual number of competing causes is a latent discrete random variable. The main interest is then in the estimation of the cured proportion as well as in developing inference about failure times of the susceptibles. The existing literature consists of parametric and non/semi-parametric approaches, while the expectation maximization (EM) algorithm offers an efficient tool for the estimation of the model parameters due to the presence of right censoring in the data. In this paper, we study the cases wherein the number of competing causes is either a binary or Poisson random variable and a piecewise linear function is used for modeling the hazard function of the time-to-event. Exact likelihood inference is then developed based on the EM algorithm and the inverse of the observed information matrix is used for developing asymptotic confidence intervals. The Monte Carlo simulation study demonstrates the accuracy of the proposed non-parametric approach compared to the results attained from the true correct parametric model. The proposed model and the inferential method is finally illustrated with a data set on cutaneous melanoma.     

双边定时截尾样本下广义逆指数分布形状参数的估计

先给出了广义逆指数分布在双边定时截尾样本下形状参数的最大似然估计,并不能得到估计的显式表达式,但证明了参数在(0,+∞)上最大似然估计是唯一存在的.其次提出用EM算法求出形状参数的估计且该估计具有良好的收敛性,还给出了形状参数的EM估计的渐近方差和近似置信区间;最后通过数值模拟,对形状参数的最大似然估计和EM估计的效果进行了比较,说明了用EM算法求形状参数的估计是可行的,并且模拟效果相对比较好.     

ML Estimation of the MultivariatetDistribution and the EM Algorithm

Maximum likelihood estimation of the multivariatetdistribution, especially with unknown degrees of freedom, has been an interesting topic in the development of the EM algorithm. After a brief review of the EM algorithm and its application to finding the maximum likelihood estimates of the parameters of thetdistribution, this paper provides new versions of the ECME algorithm for maximum likelihood estimation of the multivariatetdistribution from data with possibly missing values. The results show that the new versions of the ECME algorithm converge faster than the previous procedures. Most important, the idea of this new implementation is quite general and useful for the development of the EM algorithm. Comparisons of different methods based on two datasets are presented.     

Likelihood inference for the destructive exponentially weighted Poisson cure rate model with Weibull lifetime and an application to melanoma data

In this paper, we develop the steps of the expectation maximization algorithm (EM algorithm) for the determination of the maximum likelihood estimates (MLEs) of the parameters of the destructive exponentially weighted Poisson cure rate model in which the lifetimes are assumed to be Weibull. This model is more flexible than the promotion time cure rate model as it provides an interesting and realistic interpretation of the biological mechanism of the occurrence of an event of interest by including a destructive process of the initial number of causes in a competitive scenario. The standard errors of the MLEs are obtained by inverting the observed information matrix. An extensive Monte Carlo simulation study is carried out to evaluate the performance of the developed method of estimation. Finally, a known melanoma data are analyzed to illustrate the method of inference developed here. With these data, a comparison is also made with the scenario when the destructive mechanism is not included in the analysis.     

Initial classification of joint data in EM estimation of latent class joint model

The latent class mixture-of-experts joint model is one of the important methods for jointly modelling longitudinal and recurrent events data when the underlying population is heterogeneous and there are nonnormally distributed outcomes. The maximum likelihood estimates of parameters in latent class joint model are generally obtained by the EM algorithm. The joint distances between subjects and initial classification of subjects under study are essential to finding good starting values of the EM algorithm through formulas. In this article, separate distances and joint distances of longitudinal markers and recurrent events are proposed for classification purposes, and performance of the initial classifications based on the proposed distances and random classification are compared in a simulation study and demonstrated in an example.     

The Monte Carlo EM method for estimating multinomial probit latent variable models

We propose a multinomial probit (MNP) model that is defined by a factor analysis model with covariates for analyzing unordered categorical data, and discuss its identification. Some useful MNP models are special cases of the proposed model. To obtain maximum likelihood estimates, we use the EM algorithm with its M-step greatly simplified under Conditional Maximization and its E-step made feasible by Monte Carlo simulation. Standard errors are calculated by inverting a Monte Carlo approximation of the information matrix using Louis’s method. The methodology is illustrated with a simulated data.