Approximations to the Log-Likelihood Function in the Nonlinear Mixed-Effects Model

Abstract

Nonlinear mixed-effects models have received a great deal of attention in the statistical literature in recent years because of the flexibility they offer in handling the unbalanced repeated-measures data that arise in different areas of investigation, such as pharmacokinetics and economics. Several different methods for estimating the parameters in nonlinear mixed-effects model have been proposed. We concentrate here on two of them—maximum likelihood and restricted maximum likelihood. A rather complex numerical issue for (restricted) maximum likelihood estimation in nonlinear mixed-effects models is the evaluation of the log-likelihood function of the data, because it involves the evaluation of a multiple integral that, in most cases, does not have a closed-form expression. We consider here four different approximations to the log-likelihood, comparing their computational and statistical properties. We conclude that the linear mixed-effects (LME) approximation suggested by Lindstrom and Bates, the Laplacian approximation, and Gaussian quadrature centered at the conditional modes of the random effects are quite accurate and computationally efficient. Gaussian quadrature centered at the expected value of the random effects is quite inaccurate for a smaller number of abscissas and computationally inefficient for a larger number of abscissas. Importance sampling is accurate, but quite inefficient computationally.     

线性混合模型中方差分量的估计与QR分解

在线性混合模型中, 极大似然估计是一种很重要的估计方法, 但是它常常需要通过迭代求解. 应用设计阵的QR分解, 可以把设计阵变换成上三角矩阵. 这样可以降低参与迭代运算的矩阵的阶数, 还可以减少参与运算的数据量, 从而提高运算的速度. 本文讨论了QR分解在EM算法中的应用, 并用模拟的方法验证了QR分解可以极大的提高运算的速度. 本文同时讨论了QR分解在另外一种估计方法, 即ANOVA估计中的应用.     

广义线性回归极大似然估计的强相合性

设有该文第1节所描述的广义线性回归模型,以$\underline{\lambda}_n$和$\overline{\lambda}_n$分别记$\sum\limits_{i=1}^{n}Z_iZ_i^{\prime}$的最小和最大特征根,$\hat{\beta}_n$记$\beta_0$的极大似然估计.在文献[1]中,当\{$Z_i,i\ge1$\}有界时得到$\hat{\beta}_n$强相合的充分条件,在自然联系和非自然联系下分别为$\underline{\lambda}_n\rightarrow\infty$, $(\overline{\lambda}_n)^{1/2+\delta}=O(\underline{\lambda}_n)$（对某$\delta>0$)以及$\underline{\lambda}_n\rightarrow\infty$, $\overline{\lambda}_n=O(\underline{\lambda}_n)$.作者将后一结果改进为只要求$(\overline{\lambda}_n)^{1/2+\delta}=O(\underline{\lambda}_n)$,从而与自然联系情况下的条件达到一致.     

An Extension of Generalized Linear Models to Finite Mixture Outcome Distributions

Finite mixture distributions arise in sampling a heterogeneous population. Data drawn from such a population will exhibit extra variability relative to any single subpopulation. Statistical models based on finite mixtures can assist in the analysis of categorical and count outcomes when standard generalized linear models (GLMs) cannot adequately express variability observed in the data. We propose an extension of GLMs where the response follows a finite mixture distribution and the regression of interest is linked to the mixture’s mean. This approach may be preferred over a finite mixture of regressions when the population mean is of interest; here, only one regression must be specified and interpreted in the analysis. A technical challenge is that the mixture’s mean is a composite parameter that does not appear explicitly in the density. The proposed model maintains its link to the regression through a certain random effects structure and is completely likelihood-based. We consider typical GLM cases where means are either real-valued, constrained to be positive, or constrained to be on the unit interval. The resulting model is applied to two example datasets through Bayesian analysis. Supporting the extra variation is seen to improve residual plots and produce widened prediction intervals reflecting the uncertainty. Supplementary materials for this article are available online.     

Parameter Estimation in Hidden Markov Models With Intractable Likelihoods Using Sequential Monte Carlo

We propose sequential Monte Carlo-based algorithms for maximum likelihood estimation of the static parameters in hidden Markov models with an intractable likelihood using ideas from approximate Bayesian computation. The static parameter estimation algorithms are gradient-based and cover both offline and online estimation. We demonstrate their performance by estimating the parameters of three intractable models, namely the α-stable distribution, g-and-k distribution, and the stochastic volatility model with α-stable returns, using both real and synthetic data.     

A Computationally Efficient Projection-Based Approach for Spatial Generalized Linear Mixed Models

Inference for spatial generalized linear mixed models (SGLMMs) for high-dimensional non-Gaussian spatial data is computationally intensive. The computational challenge is due to the high-dimensional random effects and because Markov chain Monte Carlo (MCMC) algorithms for these models tend to be slow mixing. Moreover, spatial confounding inflates the variance of fixed effect (regression coefficient) estimates. Our approach addresses both the computational and confounding issues by replacing the high-dimensional spatial random effects with a reduced-dimensional representation based on random projections. Standard MCMC algorithms mix well and the reduced-dimensional setting speeds up computations per iteration. We show, via simulated examples, that Bayesian inference for this reduced-dimensional approach works well both in terms of inference as well as prediction; our methods also compare favorably to existing “reduced-rank” approaches. We also apply our methods to two real world data examples, one on bird count data and the other classifying rock types. Supplementary material for this article is available online.     

Penalized Estimation in Large-Scale Generalized Linear Array Models

Large-scale generalized linear array models (GLAMs) can be challenging to fit. Computation and storage of its tensor product design matrix can be impossible due to time and memory constraints, and previously considered design matrix free algorithms do not scale well with the dimension of the parameter vector. A new design matrix free algorithm is proposed for computing the penalized maximum likelihood estimate for GLAMs, which, in particular, handles nondifferentiable penalty functions. The proposed algorithm is implemented and available via the R package glamlasso. It combines several ideas—previously considered separately—to obtain sparse estimates while at the same time efficiently exploiting the GLAM structure. In this article, the convergence of the algorithm is treated and the performance of its implementation is investigated and compared to that of glmnet on simulated as well as real data. It is shown that the computation time for glamlasso scales favorably with the size of the problem when compared to glmnet. Supplementary materials, in the form of R code, data and visualizations of results, are available online.     

软件可靠性M-O模型和逆线性模型的参数估计

Musa-Okumoto模型和逆线性模型是研究软件可靠性的重要模型，给出了在分组数据下，M－O模型和逆线性模型中参数的最大似然估计及其存在的充分性条件，指出了[4]中的错误，并且给出了一个实例。     

线性回归模型参数的泛BC估计

本在献[1]的基础上，提出了线性回归模型参数的泛BC估计，把有偏压缩估计类的压缩系数由一维推广到多维，即由常数推广到矩阵向量，从而推出其一些优良性质。     

Efficient Mean Estimation in Log-Normal Linear Models with First-Order Correlated Errors

In this paper, we propose a log-normal linear model whose errors are first-order correlated, and suggest a two-stage method for the efficient estimation of the conditional mean of the response variable at the original scale. We obtain two estimators which minimize the asymptotic mean squared error (MM) and the asymptotic bias (MB), respectively. Both the estimators are very easy to implement, and simulation studies show that they are perform better.     

序约束下正态总体均值和方差极大似然估计的数值解法

本文讨论了k个独立正态总体的均值和方差同时在简单半序约束下的极大似然估计的求解问题,给出了计算极大估计的迭代算法,且证明该算法收敛。     

缺失数据下含几何分布的对数线性模型的EM算法

本文研究缺失数据下对数线性模型参数的极大似然估计问题.通过Monte-Carlo EM算法去拟合所提出的模型.其中,在期望步中利用Metropolis-Hastings算法产生一个缺失数据的样本,在最大化步中利用Newton-Raphson迭代使似然函数最大化.最后,利用观测数据的Fisher信息得到参数极大似然估计的渐近方差和标准误差.     

最大似然方法和Bayes方法在结构方程模型分析中的讨论(英文)

本文通过模拟研究,讨论了最大似然方法和Bayes方法在分析结构方程模型中的相似点和不同之处。     

Estimation of Smooth Time-Varying Parameters in State Space Models

In this article, we propose a penalized likelihood method to estimate time-varying parameters in standard linear state space models. The time-varying parameter is modeled as a smoothing spline and then expressed as a state space model. The maximum likelihood method is used to estimate the smoothing parameter. The proposed method is assessed by a simulation study and applied to virological response data from an HIV-infected patient receiving antiretroviral treatment.     

A Note on the Estimation of Semiparametric Two-Sample Density Ratio Models

In this paper,a semiparametric two-sample density ratio model is considered and the empirical likelihood method is applied to obtain the parameters estimation.A commonly occurring problem in computing is that the empirical likelihood function may be a concaveconvex function.Here a simple Lagrange saddle point algorithm is presented for computing the saddle point of the empirical likelihood function when the Lagrange multiplier has no explicit solution.So we can obtain the maximum empirical likelihood estimation (MELE) of parameters.Monte Carlo simulations are presented to illustrate the Lagrange saddle point algorithm.     

半参数两样本密度比率模型估计问题的一点注记

In this paper,a semiparametric two-sample density ratio model is considered and the empirical likelihood method is applied to obtain the parameters estimation.A commonly occurring problem in computing is that the empirical likelihood function may be a concaveconvex function.Here a simple Lagrange saddle point algorithm is presented for computing the saddle point of the empirical likelihood function when the Lagrange multiplier has no explicit solution.So we can obtain the maximum empirical likelihood estimation (MELE) of parameters.Monte Carlo simulations are presented to illustrate the Lagrange saddle point algorithm.     

Robust Estimation of the Generalized Pareto Distribution

One approach used for analyzing extremes is to fit the excesses over a high threshold by a generalized Pareto distribution. For the estimation of the shape and scale parameters in the generalized Pareto distribution, under some restrictions on the value of the scale parameter, maximum likelihood, method of moments and probability weighted moments' estimators are available. However, these are not robust estimators. In this paper we implement a robust estimation procedure known as the method of medians (He and Fung, 1999) to estimate the parameters in the generalized Pareto distribution. The asymptotic distribution of our estimator is normal for any value of the shape parameter except –1.     

用蒙特卡罗方法求取广义线性混合模型之最大似然估计:应用于求取乳癌死亡率之小区域估计

本文使用蒙特卡罗方法, 求得广义线性混合模型之最大似然估计, 并提供用来评估统计参数之收敛和精确度之实用方法\bd 仿真研究显示无偏之固定效应参数估计, 而方差分量估计之误差则相近于前人结果\bd 应用举例为使用泊松分布求取乳癌死亡率之小区域估计.     

A reliability application of a mixture of inverse gaussian distributions

A mixture of inverse Gaussian distributions (IGDs) is examined as a model for the lifetime of components. The components differ in one of three ways: in their initial quality, rate of wear, or variability of wear. These three cases are well represented by the parameters of the IGD model. The mechanistic interpretation of the IGD as the first passage time of Brownian motion with positive drift is adopted. The parameters considered are either dichotomous or continuous random variables. Parameter estimation is also examined for these two cases. The model seems to be most appropriate when the single IGD model fails due to heterogeneity of the initial component quality.     

带线性约束的线性模型中回归系数的平衡LS估计

基于平衡损失的思想和最小二乘统一理论,对带线性约束的一般线性模型提出了一种全面度量估计优良性的标准．给出了此标准下模型中回归系数线性函数的约束广义平衡LS估计,并得到了约束广义平衡LS估计唯一性的一个充分条件．