The restricted EM algorithm under inequality restrictions on the parameters

One of the most powerful algorithms for maximum likelihood estimation for many incomplete-data problems is the EM algorithm. The restricted EM algorithm for maximum likelihood estimation under linear restrictions on the parameters has been handled by Kim and Taylor (J. Amer. Statist. Assoc. 430 (1995) 708-716). This paper proposes an EM algorithm for maximum likelihood estimation under inequality restrictions A₀β?0, where β is the parameter vector in a linear model W=Xβ+ε and ε is an error variable distributed normally with mean zero and a known or unknown variance matrix Σ>0. Some convergence properties of the EM sequence are discussed. Furthermore, we consider the consistency of the restricted EM estimator and a related testing problem.     

基于EM算法隐Markov模型参数估计

描述最大似然参数估计问题,介绍如何用EM算法求解最大似然参数估计.首先给出EM算法的抽象形式,然后介绍EM算法的一个应用:求隐Markov模型中的参数估计.用EM算法推导出隐Markov模型中参数的迭代公式.     

基于删失数据的指数威布尔分布最大似然估计的新算法

本文讨论了指数威布尔分布当观测数据是删失数据情形时参数的最大似然估计问题.因为删失数据是一种不完全数据,我们利用EM算法来计算参数的近似最大似然估计.由于EM算法计算的复杂性,计算效率也不理想.为了克服牛顿-拉普森算法和EM算法的局限性,我们提出了一种新的方法.这种方法联合了指数威布尔分布到指数分布的变换和等效寿命数据的技巧,比牛顿-拉普森算法和EM算法更具有操作性.数据模拟讨论了这一方法的可行性.为了演示本文的方法,我们还提供了一个真实寿命数据分析的例子.     

ML Estimation of the MultivariatetDistribution and the EM Algorithm

Maximum likelihood estimation of the multivariatetdistribution, especially with unknown degrees of freedom, has been an interesting topic in the development of the EM algorithm. After a brief review of the EM algorithm and its application to finding the maximum likelihood estimates of the parameters of thetdistribution, this paper provides new versions of the ECME algorithm for maximum likelihood estimation of the multivariatetdistribution from data with possibly missing values. The results show that the new versions of the ECME algorithm converge faster than the previous procedures. Most important, the idea of this new implementation is quite general and useful for the development of the EM algorithm. Comparisons of different methods based on two datasets are presented.     

基于区间数据最大似然估计的新方法

In this paper, we study the two-parameter maximum likelihood estimation (MLE)problem for the GE distribution with consideration of interval data. In the presence of interval data, the analytical forms for the restricted MLE of the parameters of GE distribution do not exist. Since interval data is kind of incomplete data, the EM algorithm can be applied to compute the MLEs of the parameters. However the EM algorithm could be less effective.To improve effectiveness, an equivalent lifetime method is employed. The two methods are discussed via simulation studies.     

Online EM Algorithm for Hidden Markov Models

Online (also called “recursive” or “adaptive”) estimation of fixed model parameters in hidden Markov models is a topic of much interest in times series modeling. In this work, we propose an online parameter estimation algorithm that combines two key ideas. The first one, which is deeply rooted in the Expectation-Maximization (EM) methodology, consists in reparameterizing the problem using complete-data sufficient statistics. The second ingredient consists in exploiting a purely recursive form of smoothing in HMMs based on an auxiliary recursion. Although the proposed online EM algorithm resembles a classical stochastic approximation (or Robbins–Monro) algorithm, it is sufficiently different to resist conventional analysis of convergence. We thus provide limited results which identify the potential limiting points of the recursion as well as the large-sample behavior of the quantities involved in the algorithm. The performance of the proposed algorithm is numerically evaluated through simulations in the case of a noisily observed Markov chain. In this case, the algorithm reaches estimation results that are comparable to those of the maximum likelihood estimator for large sample sizes. The supplemental material for this article available online includes an appendix with the proofs of Theorem 1 and Corollary 1 stated in Section 4 as well as the MATLAB/OCTAVE code used to implement the algorithm in the case of a noisily observed Markov chain considered in Section 5.     

Coupling stochastic EM and approximate Bayesian computation for parameter inference in state-space models

We study the class of state-space models and perform maximum likelihood estimation for the model parameters. We consider a stochastic approximation expectation–maximization (SAEM) algorithm to maximize the likelihood function with the novelty of using approximate Bayesian computation (ABC) within SAEM. The task is to provide each iteration of SAEM with a filtered state of the system, and this is achieved using an ABC sampler for the hidden state, based on sequential Monte Carlo methodology. It is shown that the resulting SAEM-ABC algorithm can be calibrated to return accurate inference, and in some situations it can outperform a version of SAEM incorporating the bootstrap filter. Two simulation studies are presented, first a nonlinear Gaussian state-space model then a state-space model having dynamics expressed by a stochastic differential equation. Comparisons with iterated filtering for maximum likelihood inference, and Gibbs sampling and particle marginal methods for Bayesian inference are presented.     

Parameter estimation in nonlinear stochastic differential equations

We discuss the problem of parameter estimation in nonlinear stochastic differential equations (SDEs) based on sampled time series. A central message from the theory of integrating SDEs is that there exist in general two time scales, i.e. that of integrating these equations and that of sampling. We argue that therefore, maximum likelihood estimation is computationally extremely expensive. We discuss the relation between maximum likelihood and quasi maximum likelihood estimation. In a simulation study, we compare the quasi maximum likelihood method with an approach for parameter estimation in nonlinear SDEs that disregards the existence of the two time scales.     

Computationally Efficient Estimation for the Generalized Odds Rate Mixture Cure Model With Interval-Censored Data

For semiparametric survival models with interval-censored data and a cure fraction, it is often difficult to derive nonparametric maximum likelihood estimation due to the challenge in maximizing the complex likelihood function. In this article, we propose a computationally efficient EM algorithm, facilitated by a gamma-Poisson data augmentation, for maximum likelihood estimation in a class of generalized odds rate mixture cure (GORMC) models with interval-censored data. The gamma-Poisson data augmentation greatly simplifies the EM estimation and enhances the convergence speed of the EM algorithm. The empirical properties of the proposed method are examined through extensive simulation studies and compared with numerical maximum likelihood estimates. An R package “GORCure” is developed to implement the proposed method and its use is illustrated by an application to the Aerobic Center Longitudinal Study dataset. Supplementary material for this article is available online.     

Likelihood Inference for Large Scale Stochastic Blockmodels With Covariates Based on a Divide-and-Conquer Parallelizable Algorithm With Communication

We consider a stochastic blockmodel equipped with node covariate information, that is, helpful in analyzing social network data. The key objective is to obtain maximum likelihood estimates of the model parameters. For this task, we devise a fast, scalable Monte Carlo EM type algorithm based on case-control approximation of the log-likelihood coupled with a subsampling approach. A key feature of the proposed algorithm is its parallelizability, by processing portions of the data on several cores, while leveraging communication of key statistics across the cores during each iteration of the algorithm. The performance of the algorithm is evaluated on synthetic datasets and compared with competing methods for blockmodel parameter estimation. We also illustrate the model on data from a Facebook derived social network enhanced with node covariate information. Supplemental materials for this article are available online.     

Problems with the estimation of stochastic differential equations using structural equations models

The present paper deals with the identification and maximum likelihood estimation of systems of linear stochastic differential equations using panel data. So we only have a sample of discrete observations over time of the relevant variables for each individual. A popular approach in the social sciences advocates the estimation of the “exact discrete model” after a reparameterization with LISREL or similar programs for structural equations models. The “exact discrete model” corresponds to the continuous time model in the sense that observations at equidistant points in time that are generated by the latter system also satisfy the former. In the LISREL approach the reparameterized discrete time model is estimated first without taking into account the nonlinear mapping from the continuous to the discrete time parameters. In a second step, using the inverse mapping, the fundamental system parameters of the continuous time system in which we are interested, are inferred. However, some severe problems arise with this “indirect approach”. First, an identification problem may arise in multiple equation systems, since the matrix exponential function denning some of the new parameters is in general not one‐to‐one, and hence the inverse mapping mentioned above does not exist. Second, usually some sort of approximation of the time paths of the exogenous variables is necessary before the structural parameters of the system can be estimated with discrete data. Two simple approximation methods are discussed. In both approximation methods the resulting new discrete time parameters are connected in a complicated way. So estimating the reparameterized discrete model by OLS without restrictions does not yield maximum likelihood estimates of the desired continuous time parameters as claimed by some authors. Third, a further limitation of estimating the reparameterized model with programs for structural equations models is that even simple restrictions on the original fundamental parameters of the continuous time system cannot be dealt with. This issue is also discussed in some detail. For these reasons the “indirect method” cannot be recommended. In many cases the approach leads to misleading inferences. We strongly advocate the direct estimation of the continuous time parameters. This approach is more involved, because the exact discrete model is nonlinear in the original parameters. A computer program by Hermann Singer that provides appropriate maximum likelihood estimates is described.     

双边定时截尾样本下广义逆指数分布形状参数的估计

先给出了广义逆指数分布在双边定时截尾样本下形状参数的最大似然估计,并不能得到估计的显式表达式,但证明了参数在(0,+∞)上最大似然估计是唯一存在的.其次提出用EM算法求出形状参数的估计且该估计具有良好的收敛性,还给出了形状参数的EM估计的渐近方差和近似置信区间;最后通过数值模拟,对形状参数的最大似然估计和EM估计的效果进行了比较,说明了用EM算法求形状参数的估计是可行的,并且模拟效果相对比较好.     

Nonlinear Smoothing and the EM Algorithm for Positive Integral Equations of the First Kind

We study a modification of the EMS algorithm in which each step of the EMS algorithm is preceded by a nonlinear smoothing step of the form , where S is the smoothing operator of the EMS algorithm. In the context of positive integral equations (à la positron emission tomography) the resulting algorithm is related to a convex minimization problem which always admits a unique smooth solution, in contrast to the unmodified maximum likelihood setup. The new algorithm has slightly stronger monotonicity properties than the original EM algorithm. This suggests that the modified EMS algorithm is actually an EM algorithm for the modified problem. The existence of a smooth solution to the modified maximum likelihood problem and the monotonicity together imply the strong convergence of the new algorithm. We also present some simulation results for the integral equation of stereology, which suggests that the new algorithm behaves roughly like the EMS algorithm. Accepted 1 April 1997     

一类零膨胀半参数混合效应模型的惩罚似然估计

本文将半参数线性混合效应模型推广应用到一类具有零膨胀的纵向数据或集群数据的研究中,提出了一类新的半参数混合效应模型,然后利用广义交叉核实法选取光滑参数,通过最大惩罚似然函数方法与EM算法给出了模型参数部分与非参数部分的估计方法,最后,通过模拟和实例说明了本文方法的有效性.     

Parameter estimation for Hammerstein CARARMA systems based on the Newton iteration

The Newton iteration is basic for solving nonlinear optimization problems and studying parameter estimation algorithms. In this letter, a maximum likelihood estimation algorithm is developed for estimating the parameters of Hammerstein nonlinear controlled autoregressive autoregressive moving average (CARARMA) systems by using the Newton iteration. A simulation example is provided to show the effectiveness of the proposed algorithm.     

正态分布和瑞利分布混合情形下的参数估计及分类问题

学者往往用单一的分布模拟和拟合杂波,如正态分布、瑞利分布和威布尔分布等。然而在实际中,雷达杂波由多种类型的杂波组成,单一分布通常不能精确刻画雷达杂波规律,因此,应用混合分布模型对雷达杂波数据建模更准确。本文考虑用正态分布和瑞利分布的混合分布拟合杂波,并应用矩估计方法和基于EM算法的极大似然估计方法估计模型参数,最后,应用最大后验概率分类准则验证2种估计方法的分类准确率。通过数据模拟,得出极大似然估计的效果和分类准确率都要优于矩估计的估计效果和分类准确率。     

Estimation of parameters in latent class models using fuzzy clustering algorithms

A mixture approach to clustering is an important technique in cluster analysis. A mixture of multivariate multinomial distributions is usually used to analyze categorical data with latent class model. The parameter estimation is an important step for a mixture distribution. Described here are four approaches to estimating the parameters of a mixture of multivariate multinomial distributions. The first approach is an extended maximum likelihood (ML) method. The second approach is based on the well-known expectation maximization (EM) algorithm. The third approach is the classification maximum likelihood (CML) algorithm. In this paper, we propose a new approach using the so-called fuzzy class model and then create the fuzzy classification maximum likelihood (FCML) approach for categorical data. The accuracy, robustness and effectiveness of these four types of algorithms for estimating the parameters of multivariate binomial mixtures are compared using real empirical data and samples drawn from the multivariate binomial mixtures of two classes. The results show that the proposed FCML algorithm presents better accuracy, robustness and effectiveness. Overall, the FCML algorithm has the superiority over the ML, EM and CML algorithms. Thus, we recommend FCML as another good tool for estimating the parameters of mixture multivariate multinomial models.     

有关顺序统计量的极大似然估计

讨论了一类参数空间受样本限制的极大似然估计问题.分析了随机变量分布的非零区域与似然函数定义域的对应关系,提出如果分布的非零区域受参数限制,则无论似然方程是否可解,参数的极大似然估计必然与样本顺序统计量X_((n))或X_((1))有关,并具体分析了似然估计一定等于、一定不等于和可能等于顺序统计量X_((n))(X_((1)))的三种情形,并给出了相应的判别条件.最后分析得出在第三种判别条件之下,似然估计是否取值于x_((n))(x_((1)))视具体的样本观测值决定.     

���ɾʧ�����»��ָ���ֲ��Ĺ���

The hybrid censoring scheme is a mixture of type-I and type-II censoring schemes. It is a popular censoring scheme in the literature of life data analysis. Mixed exponential distribution (MED) models is a class of favorable models in reliability statistics. Nevertheless, there is no much discussion to focus on parameters estimation for MED models with hybrid censored samples. We will address this problem in this paper. The EM (Expectation-Maximization) algorithm is employed to derive the closed form of the maximum likelihood estimators (MLEs). Finally, Monte Carlo simulations and a real-world data analysis are conducted to illustrate the proposed method.     

半参数两样本密度比率模型估计问题的一点注记

In this paper,a semiparametric two-sample density ratio model is considered and the empirical likelihood method is applied to obtain the parameters estimation.A commonly occurring problem in computing is that the empirical likelihood function may be a concaveconvex function.Here a simple Lagrange saddle point algorithm is presented for computing the saddle point of the empirical likelihood function when the Lagrange multiplier has no explicit solution.So we can obtain the maximum empirical likelihood estimation (MELE) of parameters.Monte Carlo simulations are presented to illustrate the Lagrange saddle point algorithm.