Coupling stochastic EM and approximate Bayesian computation for parameter inference in state-space models

We study the class of state-space models and perform maximum likelihood estimation for the model parameters. We consider a stochastic approximation expectation–maximization (SAEM) algorithm to maximize the likelihood function with the novelty of using approximate Bayesian computation (ABC) within SAEM. The task is to provide each iteration of SAEM with a filtered state of the system, and this is achieved using an ABC sampler for the hidden state, based on sequential Monte Carlo methodology. It is shown that the resulting SAEM-ABC algorithm can be calibrated to return accurate inference, and in some situations it can outperform a version of SAEM incorporating the bootstrap filter. Two simulation studies are presented, first a nonlinear Gaussian state-space model then a state-space model having dynamics expressed by a stochastic differential equation. Comparisons with iterated filtering for maximum likelihood inference, and Gibbs sampling and particle marginal methods for Bayesian inference are presented.     

Divide-and-Conquer With Sequential Monte Carlo

We propose a novel class of Sequential Monte Carlo (SMC) algorithms, appropriate for inference in probabilistic graphical models. This class of algorithms adopts a divide-and-conquer approach based upon an auxiliary tree-structured decomposition of the model of interest, turning the overall inferential task into a collection of recursively solved subproblems. The proposed method is applicable to a broad class of probabilistic graphical models, including models with loops. Unlike a standard SMC sampler, the proposed divide-and-conquer SMC employs multiple independent populations of weighted particles, which are resampled, merged, and propagated as the method progresses. We illustrate empirically that this approach can outperform standard methods in terms of the accuracy of the posterior expectation and marginal likelihood approximations. Divide-and-conquer SMC also opens up novel parallel implementation options and the possibility of concentrating the computational effort on the most challenging subproblems. We demonstrate its performance on a Markov random field and on a hierarchical logistic regression problem. Supplementary materials including proofs and additional numerical results are available online.     

Bayesian Inference in Hidden Markov Random Fields for Binary Data Defined on Large Lattices

Hidden Markov random fields represent a complex hierarchical model, where the hidden latent process is an undirected graphical structure. Performing inference for such models is difficult primarily because the likelihood of the hidden states is often unavailable. The main contribution of this article is to present approximate methods to calculate the likelihood for large lattices based on exact methods for smaller lattices. We introduce approximate likelihood methods by relaxing some of the dependencies in the latent model, and also by extending tractable approximations to the likelihood, the so-called pseudolikelihood approximations, for a large lattice partitioned into smaller sublattices. Results are presented based on simulated data as well as inference for the temporal-spatial structure of the interaction between up- and down-regulated states within the mitochondrial chromosome of the Plasmodium falciparum organism. Supplemental material for this article is available online.     

Normal Linear Regression Models With Recursive Graphical Markov Structure

A multivariate normal statistical model defined by the Markov properties determined by an acyclic digraph admits a recursive factorization of its likelihood function (LF) into the product of conditional LFs, each factor having the form of a classical multivariate linear regression model (≡WMANOVA model). Here these models are extended in a natural way to normal linear regression models whose LFs continue to admit such recursive factorizations, from which maximum likelihood estimators and likelihood ratio (LR) test statistics can be derived by classical linear methods. The central distribution of the LR test statistic for testing one such multivariate normal linear regression model against another is derived, and the relation of these regression models to block-recursive normal linear systems is established. It is shown how a collection of nonnested dependent normal linear regression models (≡Wseemingly unrelated regressions) can be combined into a single multivariate normal linear regression model by imposing a parsimonious set of graphical Markov (≡Wconditional independence) restrictions.     

基于经验似然的部分线性模型的统计诊断

经验似然方法己经被广泛应用于许多模型的统计推断.本文基于经验似然对部分线性模型进行统计诊断.首先给出模型的估计方程,进而得到模型参数的极大经验似然估计;其次,基于经验似然研究了三种不同的影响曲率;最后通过随机模拟和实例分析,说明了统计诊断方法的有效性.     

Probabilistic inference with noisy-threshold models based on a CP tensor decomposition

The specification of conditional probability tables (CPTs) is a difficult task in the construction of probabilistic graphical models. Several types of canonical models have been proposed to ease that difficulty. Noisy-threshold models generalize the two most popular canonical models: the noisy-or and the noisy-and. When using the standard inference techniques the inference complexity is exponential with respect to the number of parents of a variable. More efficient inference techniques can be employed for CPTs that take a special form. CPTs can be viewed as tensors. Tensors can be decomposed into linear combinations of rank-one tensors, where a rank-one tensor is an outer product of vectors. Such decomposition is referred to as Canonical Polyadic (CP) or CANDECOMP-PARAFAC (CP) decomposition. The tensor decomposition offers a compact representation of CPTs which can be efficiently utilized in probabilistic inference. In this paper we propose a CP decomposition of tensors corresponding to CPTs of threshold functions, exactly ℓ-out-of-k functions, and their noisy counterparts. We prove results about the symmetric rank of these tensors in the real and complex domains. The proofs are constructive and provide methods for CP decomposition of these tensors. An analytical and experimental comparison with the parent-divorcing method (which also has a polynomial complexity) shows superiority of the CP decomposition-based method. The experiments were performed on subnetworks of the well-known QMRT-DT network generalized by replacing noisy-or by noisy-threshold models.     

Nonconservative Estimating Functions and Approximate Quasi-Likelihoods

The estimating function approach unifies two dominant methodologies in statistical inferences: Gauss's least square and Fisher's maximum likelihood. However, a parallel likelihood inference is lacking because estimating functions are in general not integrable, or nonconservative. In this paper, nonconservative estimating functions are studied from vector analysis perspective. We derive a generalized version of the Helmholtz decomposition theorem for estimating functions of any dimension. Based on this theorem we propose locally quadratic potentials as approximate quasi-likelihoods. Quasi-likelihood ratio tests are studied. The ideas are illustrated by two examples: (a) logistic regression with measurement error model and (b) probability estimation conditional on marginal frequencies.     

Structural-EM for learning PDG models from incomplete data

Probabilistic Decision Graphs (PDGs) are a class of graphical models that can naturally encode some context specific independencies that cannot always be efficiently captured by other popular models, such as Bayesian Networks. Furthermore, inference can be carried out efficiently over a PDG, in time linear in the size of the model. The problem of learning PDGs from data has been studied in the literature, but only for the case of complete data. We propose an algorithm for learning PDGs in the presence of missing data. The proposed method is based on the Expectation-Maximisation principle for estimating the structure of the model as well as the parameters. We test our proposal on both artificially generated data with different rates of missing cells and real incomplete data. We also compare the PDG models learnt by our approach to the commonly used Bayesian Network (BN) model. The results indicate that the PDG model is less sensitive to the rate of missing data than BN model. Also, though the BN models usually attain higher likelihood, the PDGs are close to them also in size, which makes the learnt PDGs preferable for probabilistic inference purposes.     

Semiparametric Bayesian Regression via Potts Model

We consider Bayesian nonparametric regression through random partition models. Our approach involves the construction of a covariate-dependent prior distribution on partitions of individuals. Our goal is to use covariate information to improve predictive inference. To do so, we propose a prior on partitions based on the Potts clustering model associated with the observed covariates. This drives by covariate proximity both the formation of clusters, and the prior predictive distribution. The resulting prior model is flexible enough to support many different types of likelihood models. We focus the discussion on nonparametric regression. Implementation details are discussed for the specific case of multivariate multiple linear regression. The proposed model performs well in terms of model fitting and prediction when compared to other alternative nonparametric regression approaches. We illustrate the methodology with an application to the health status of nations at the turn of the 21st century. Supplementary materials are available online.     

��I�ͻ�Ͻ�β��ָ��——�������ֲ�����ʧЧģ�͵�ͳ�Ʒ���

In this paper, we investigate a competing risks model based on exponentiated Weibull distribution under Type-I progressively hybrid censoring scheme. To estimate the unknown parameters and reliability function, the maximum likelihood estimators and asymptotic confidence intervals are derived. Since Bayesian posterior density functions cannot be given in closed forms, we adopt Markov chain Monte Carlo method to calculate approximate Bayes estimators and highest posterior density credible intervals. To illustrate the estimation methods, a simulation study is carried out with numerical results. It is concluded that the maximum likelihood estimation and Bayesian estimation can be used for statistical inference in competing risks model under Type-I progressively hybrid censoring scheme.     

Composite likelihood and maximum likelihood methods for joint latent class modeling of disease prevalence and high-dimensional semicontinuous biomarker data

Joint latent class modeling of disease prevalence and high-dimensional semicontinuous biomarker data has been proposed to study the relationship between diseases and their related biomarkers. However, statistical inference of the joint latent class modeling approach has proved very challenging due to its computational complexity in seeking maximum likelihood estimates. In this article, we propose a series of composite likelihoods for maximum composite likelihood estimation, as well as an enhanced Monte Carlo expectation–maximization (MCEM) algorithm for maximum likelihood estimation, in the context of joint latent class models. Theoretically, the maximum composite likelihood estimates are consistent and asymptotically normal. Numerically, we have shown that, as compared to the MCEM algorithm that maximizes the full likelihood, not only the composite likelihood approach that is coupled with the quasi-Newton method can substantially reduce the computational complexity and duration, but it can simultaneously retain comparative estimation efficiency.     

Asymptotic efficiency of the two-stage estimation method for copula-based models

For multivariate copula-based models for which maximum likelihood is computationally difficult, a two-stage estimation procedure has been proposed previously; the first stage involves maximum likelihood from univariate margins, and the second stage involves maximum likelihood of the dependence parameters with the univariate parameters held fixed from the first stage. Using the theory of inference functions, a partitioned matrix in a form amenable to analysis is obtained for the asymptotic covariance matrix of the two-stage estimator. The asymptotic relative efficiency of the two-stage estimation procedure compared with maximum likelihood estimation is studied. Analysis of the limiting cases of the independence copula and Fréchet upper bound help to determine common patterns in the efficiency as the dependence in the model increases. For the Fréchet upper bound, the two-stage estimation procedure can sometimes be equivalent to maximum likelihood estimation for the univariate parameters. Numerical results are shown for some models, including multivariate ordinal probit and bivariate extreme value distributions, to indicate the typical level of asymptotic efficiency for discrete and continuous data.     

A Gaussian Mixed Model for Learning Discrete Bayesian Networks

In this paper, we address the problem of learning discrete Bayesian networks from noisy data. A graphical model based on a mixture of Gaussian distributions with categorical mixing structure coming from a discrete Bayesian network is considered. The network learning is formulated as a maximum likelihood estimation problem and performed by employing an EM algorithm. The proposed approach is relevant to a variety of statistical problems for which Bayesian network models are suitable—from simple regression analysis to learning gene/protein regulatory networks from microarray data.     

Markov modulated Poisson process associated with state-dependent marks and its applications to the deep earthquakes

In this paper, we introduce one type of Markov-Modulated Poisson Process (MMPP) whose arrival times are associated with state-dependent marks. Statistical inference problems including the derivation of the likelihood, parameter estimation through EM algorithm and statistical inference on the state process and the observed point process are addressed. A goodness-of-fit test is proposed for MMPP with state-dependent marks by utilizing the theories of rescaling marked point process. We also perform some numerical simulations to indicate the effects of different marks on the efficiencies and accuracies of MLE. The effects of the attached marks on the estimation tend to be weakened for increasing data sizes. Then we apply these methods to characterize the occurrence patterns of New Zealand deep earthquakes through a second-order MMPP with state-dependent marks. In this model, the occurrence times and magnitudes of the deep earthquakes are associated with two levels of seismicity which evolves in terms of an unobservable two-state Markov chain.     

Maximum Likelihood Estimation of Hidden Markov Multivariate Normal Distribution Parameters

Hidden Markov model is widely used in statistical modeling of time, space and state transition data. The definition of hidden Markov multivariate normal distribution is given. The principle of using cluster analysis to determine the hidden state of observed variables is introduced. The maximum likelihood estimator of the unknown parameters in the model is derived. The simulated observation data set is used to test the estimation effect and stability of the method. The characteristic is simple classical statistical inference such as cluster analysis and maximum likelihood estimation. The method solves the parameter estimation problem of complex statistical models.     

������ɷ��Ԫ��̬�ֲ������ļ�����Ȼ����

??Hidden Markov model is widely used in statistical modeling of time, space and state transition data. The definition of hidden Markov multivariate normal distribution is given. The principle of using cluster analysis to determine the hidden state of observed variables is introduced. The maximum likelihood estimator of the unknown parameters in the model is derived. The simulated observation data set is used to test the estimation effect and stability of the method. The characteristic is simple classical statistical inference such as cluster analysis and maximum likelihood estimation. The method solves the parameter estimation problem of complex statistical models.     

方差分量模型协方差阵的谱分解

本文对平衡方差分量模型, 给出了其协方差阵的新的谱分解算法. 该方法的特点是计算简单, 易于理解, 无须复杂的数学知识. 且能够明确显示协方差阵的不同特征值的个数, 以及谱分解中不同特征值所对应的投影阵的显式表示. 基于新方法我们进一步研究了平衡方差分量模型的一些相关性质.本文还研究了一般方差分量模型, 我们首先定义了一般方差分量模型协方差阵的简单谱分解,给出了一般方差分量模型可以进行简单谱分解的充要条件, 并研究了协方差阵简单谱分解的一些性质. 对于协方差阵可以进行简单谱分解的方差分量模型, 本文研究了简单谱分解在其统计推断中的应用.     

基于EMD-GA-BP与EMD-PSO-LSSVM的中国碳市场价格预测

由于碳交易市场价格的波动性大及相互影响关系的复杂性,本文试图构建碳价格长期和短期的最优预测模型。考虑到碳交易价格波动的趋势性和周期性特点,基于经验模态分解算法(EMD)、遗传算法(GA)—神经网络(BP)模型、粒子群算法(PSO)—最小二乘支持向量机(LSSVM)模型及由它们构建的组合预测模型,对中国碳市场交易价格进行短期预测和长期预测。实证分析中将影响碳交易价格的不同宏观经济因素和碳价格时间序列因素做为输入变量,分别代入组合模型进行预测。研究结果表明,在短期预测中,EMD-GA-BP模型预测效果优于GA-BP模型和PSO-LSSVM模型;而在长期预测中,组合模型EMD-PSO-LSSVM模型预测效果优于只考虑碳价格波动趋势性或周期性预测效果。     

Heteroscedastic replicated measurement error models under asymmetric heavy-tailed distributions

We propose a heteroscedastic replicated measurement error model based on the class of scale mixtures of skew-normal distributions, which allows the variances of measurement errors to vary across subjects. We develop EM algorithms to calculate maximum likelihood estimates for the model with or without equation error. An empirical Bayes approach is applied to estimate the true covariate and predict the response. Simulation studies show that the proposed models can provide reliable results and the inference is not unduly affected by outliers and distribution misspecification. The method has also been used to analyze a real data of plant root decomposition.     

Variational Bayes With Intractable Likelihood

Variational Bayes (VB) is rapidly becoming a popular tool for Bayesian inference in statistical modeling. However, the existing VB algorithms are restricted to cases where the likelihood is tractable, which precludes their use in many interesting situations such as in state--space models and in approximate Bayesian computation (ABC), where application of VB methods was previously impossible. This article extends the scope of application of VB to cases where the likelihood is intractable, but can be estimated unbiasedly. The proposed VB method therefore makes it possible to carry out Bayesian inference in many statistical applications, including state--space models and ABC. The method is generic in the sense that it can be applied to almost all statistical models without requiring too much model-based derivation, which is a drawback of many existing VB algorithms. We also show how the proposed method can be used to obtain highly accurate VB approximations of marginal posterior distributions. Supplementary material for this article is available online.