非随机缺失机制下基于模型的参数似然估计方法研究

在纵向抽样调查活动中,常出现变量数据缺失的情况,如何对含缺失的数据集进行总体参数估计是一个热点话题。目前已有方法主要适用于随机缺失机制下的缺失数据分析问题,常采用插补法生成完整数据集,基于此进行参数估计。本文在非随机数据缺失机制下,研究了几种基于模型的参数似然估计方法,包括模式混合模型法和选择模型法,对单调缺失模式下含缺失纵向调查数据给出了参数估计范例,进而引入随机效应参数,将两种方法加以推广。     

一个贝叶斯聚类方法及应用

聚类分析是研究对样品或指标进行综合分类的一种多元统计分析方法.聚类结果常表现为树状图的形式.如何合理确定聚类的个数,一直是一个比较困难的问题,至今没有很好的解决方案,尤其当样本量较大时,树状图层次较多,很难直观确定聚类个数.介绍一种基于贝叶斯理论的聚类方法,通过对后验似然最大化的原则确定最佳聚类个数和方案,避免了聚类个数选择的主观性.一个已知分类情况的实际数据验证了该聚类方法的有效性.     

基于似然函数的纵向数据线性混合模型影响分析

线性混合模型在纵向数据分析中有广泛应用,应用点删除法研究纵向数据线性混合模型影响分析问题.采用的影响度量是基于似然函数构造的广义Cook距离,推导出两种广义Cook距离并用于实际数据分析和统计模拟分析,实际数据分析和统计模拟分析结果都说明该方法是行之有效的.     

带协变量的负二项回归模型中离散参数估计的研究

针对带协变量的负二项回归模型中离散参数估计问题,推广了极大似然估计和Bootstrap极大似然估计方法,并在绝对偏差的意义下,通过模拟研究和实际数据分析研究了估计的优良性.研究结果表明协变量和样本量均对离散参数估计有影响.     

带协变量的负二项回归模型中离散参数估计的研究（英文）

针对带协变量的负二项回归模型中离散参数估计问题,推广了极大似然估计和Bootstrap极大似然估计方法,并在绝对偏差的意义下,通过模拟研究和实际数据分析研究了估计的优良性.研究结果表明协变量和样本量均对离散参数估计有影响.     

混合Weibull分布参数估计的ECM算法

混合威布尔分布是寿命数据分析中一个重要的统计模型.但是利用传统的统计方法,如矩估计、极大似然估计等估计模型的参数比较困难.应用ECM算法详细研究了混合威布尔分布在正常工作条件下,完全数据场合、Ⅰ-型截尾和Ⅱ-截尾场合的参数估计问题.数据模拟表明利用ECM算法来估计混合威布尔分布是一种有效的方法.     

零膨胀泊松混合回归中的变量选择

来源于不同总体的数据异质性较大,数据“零取值”较多且离散度大,可利用零膨胀泊松(ZIP)混合回归模型建模分析,然而混合模型中自变量较多.为了筛选出重要变量,本文利用自适应LASSO对ZIP混合回归模型进行变量选择,即在似然函数中加入惩罚项,再利用EM算法估计参数.通过模拟,验证了该方法在变量选择和参数估计中的有效性.同时,将ZIP混合回归模型应用于预测借贷失败次数的实际数据分析,筛选出对借贷失败有重要影响的因素.最后,通过比较各模型的预测效果,得到ZIP混合回归模型优于泊松(Poisson),负二项(NB)和ZIP回归模型.     

偏正态混合模型的惩罚极大似然估计

在分析具有异质性和非对称性数据时,偏正态混合模型提供一种比经典的Gauss混合模型更为灵活的建模方式.然而,由于无界的似然函数和发散的形状参数,该模型的极大似然估计并未被正确定义,进一步导致不理想的推断过程.为同时解决这两个问题,本文基于惩罚似然提出一种新的估计方案,并证明在混合分布的类别个数大于或等于真实的类别个数时,相应的惩罚极大似然估计是强相合的.同时,本文也提出相应的惩罚EM (expectation maximization)算法来计算惩罚估计.最后,通过模拟分析与现有方法比较研究估计方法在有限样本下的表现,并采用两个实例说明方法的有效性.     

周期数据时段划分的双截断高斯混合模型及其EM算法

有限混合模型是多模态数据拟合和聚类的有力工具,本文针对具有多模态的周期数据提出了双截断高斯混合糢型,并推导出相应的EM算法,再通过BIC准則确定混合成分个数,该方法的优点是可以将相邻周期上距离较近的数据聚为一类.模拟研究显示,在具体参数设置下,EM算法和BIC准则是相合的。最后,该方法应用于车流量数据的时段划分,将一天划分为具有显著特征的6个时段,有助于交通部门采取相应策略,为优化交通灯信号配时提供参考依据.     

居民收入分布函数的实证研究

收入分布函数的估计方法主要有参数估计法与非参数估计的方法.利用参数估计方法,依据黑龙江省及国家城镇居民人均可支配收入数据,分别采用极大似然法与分段计算分布总体中的参数,确定收入分布函数,然后根据分布函数与实际数据的拟合状况,验证黑龙江省及国家人均可支配收入服从对数正态分布,但是参数的确定方式决定了拟合的有效性.     

基于Dirichlet分布有限混合模型的Bayes聚类

本文基于Dirichlet分布有限混合模型,提出了一种用于成分数据的Bayes聚类方法.采用EM算法获得模型参数的估计,用BIC准则确定类数,用类似于Bayes判别的方法对各观测分类.推导了计算公式,编写出程序.模拟研究结果表明,本文提出的方法有较好的聚类效果.     

变量的有限混合模型对有序数据的Bayes聚类分析

本文基于隐变量的有限混合模型, 提出了一种用于有序数据的Bayes聚类方法\bd 我们采用EM算法获得模型参数的估计, 用BIC准则确定类数, 用类似于Bayes判别的方法对各观测分类\bd 模拟研究结果表明, 本文提出的方法有较好的聚类效果, 对于中等规模的数据集, 计算量是可以接受的.     

On modeling left-truncated loss data using mixtures of distributions

A new statistical methodology is developed for fitting left-truncated loss data by using the G-component finite mixture model with any combination of Gamma, Lognormal, and Weibull distributions. The EM algorithm, along with the emEM initialization strategy, is employed for model fitting. We propose a new grid map which considers the model selection criterion (AIC or BIC) and risk measures at the same time, by using the entire space of models under consideration. A simulation study validates our proposed approach. The application of the proposed methodology and use of new grid maps are illustrated through analyzing a real data set that includes left-truncated insurance losses.     

厄兰极值混合模型的有效估计及其在保险中的应用

本文研究了Erlang混合分布和广义帕累托分布混合模型的估计问题.通过引入iSCAD惩罚函数,利用EM算法极大化iSCAD惩罚似然函数的方法,获得了混合序和参数的估计值,计算出有效的度量风险指标value-at-risk(VaR)和tail-VaR(TVaR),通过模拟实验和实际数据说明了模型和算法的有效性.推广了有限Erlang极值混合模型在保险数据拟合中的应用.     

Towards data driven selection of a penalty function for data driven Neyman tests

The data driven Neyman statistic consists of two elements: a score statistic in a finite dimensional submodel and a selection rule to determine the best fitted submodel. For instance, Schwarz BIC and Akaike AIC rules are often applied in such constructions. For moderate sample sizes AIC is sensitive in detecting complex models, while BIC works well for relatively simple structures. When the sample size is moderate, the choice of selection rule for determining a best fitted model from a number of models has a substantial influence on the power of the related data driven Neyman test. This paper proposes a new solution, in which the type of penalty (AIC or BIC) is chosen on the basis of the data. The resulting refined data driven test combines the advantages of these two selection rules.     

Statistical estimation of a mixture of Gaussian distributions

The paper is devoted to the problem of statistical estimation of a multivariate distribution density, which is a discrete mixture of Gaussian distributions. A heuristic approach is considered, based on the use of the EM algorithm and nonparametric density estimation with a sequential increase in the number of components of the mixture. Criteria for testing of model adequacy are discussed.     

A strategy for analysing mixed and pooled exponentials

It is shown that a linear plot of the mean residual life on the failure rate characterizes the mixture of two exponentials. This plot is used to estimate the two components in the mixing distribution with the two largest mixing proportions. The EM algorithm is then used with these as initial values to obtain the MLE. Gradient plots are used to see if a higher-order fit is needed. A heuristic is given on how to use the gradient plots to identify components in the higher-order fit when this is the case. Graphs of an assignment function are then used to determine if the data are from a mixed model or simply the effect of pooling.     

An EM-Like Algorithm for Semi- and Nonparametric Estimation in Multivariate Mixtures

We propose an algorithm for nonparametric estimation for finite mixtures of multivariate random vectors that strongly resembles a true EM algorithm. The vectors are assumed to have independent coordinates conditional upon knowing from which mixture component they come, but otherwise their density functions are completely unspecified. Sometimes, the density functions may be partially specified by Euclidean parameters, a case we call semiparametric. Our algorithm is much more flexible and easily applicable than existing algorithms in the literature; it can be extended to any number of mixture components and any number of vector coordinates of the multivariate observations. Thus it may be applied even in situations where the model is not identifiable, so care is called for when using it in situations for which identifiability is difficult to establish conclusively. Our algorithm yields much smaller mean integrated squared errors than an alternative algorithm in a simulation study. In another example using a real dataset, it provides new insights that extend previous analyses. Finally, we present two different variations of our algorithm, one stochastic and one deterministic, and find anecdotal evidence that there is not a great deal of difference between the performance of these two variants. The computer code and data used in this article are available online.