Probability estimation via smoothing in sparse contingency tables with ordered categories

Probability estimation in sparse two-dimensional contingency tables with ordered categories is examined. Several smoothing procedures are compared to analysis of the unsmoothed table. It is shown that probability estimates obtained via maximum penalized likelihood smoothing are consistent under a sparse asymptotic framework if the underlying probability matrix is smooth, and are more accurate than kernel-based and other smoothing techniques. In fact, computer simulations indicate that smoothing based on a product kernel is less effective than no smoothing at all. An example is given to illustrate the smoothing technique. Possible extensions to model building and higher dimensional tables are discussed.     

Optimal scaling for association models when category scores have a natural ordering

Association models considered by Goodman (1979) for the analysis of contingency tables with ordered categories can provide the scores of row and/or column categories. If there exist natural ordering among the scores in the population, the estimated scores would be required to coincide with the prescribed order.In this article, we propose the methods for deriving maximum likelihood (ML) estimates when the category scores are subject to restrictions.     

Context-specific independence in graphical log-linear models

Log-linear models are the popular workhorses of analyzing contingency tables. A log-linear parameterization of an interaction model can be more expressive than a direct parameterization based on probabilities, leading to a powerful way of defining restrictions derived from marginal, conditional and context-specific independence. However, parameter estimation is often simpler under a direct parameterization, provided that the model enjoys certain decomposability properties. Here we introduce a cyclical projection algorithm for obtaining maximum likelihood estimates of log-linear parameters under an arbitrary context-specific graphical log-linear model, which needs not satisfy criteria of decomposability. We illustrate that lifting the restriction of decomposability makes the models more expressive, such that additional context-specific independencies embedded in real data can be identified. It is also shown how a context-specific graphical model can correspond to a non-hierarchical log-linear parameterization with a concise interpretation. This observation can pave way to further development of non-hierarchical log-linear models, which have been largely neglected due to their believed lack of interpretability.     

Decomposition of Covariate-Dependent Graphical Models with Categorical Data

Graphical models are wildly used to describe conditional dependence relationships among interacting random variables. Among statistical inference problems of a graphical model, one particular interest is utilizing its interaction structure to reduce model complexity. As an important approach to utilizing structural information, decomposition allows a statistical inference problem to be divided into some sub-problems with lower complexities. In this paper, to investigate decomposition of covariate-dependent graphical models, we propose some useful definitions of decomposition of covariate-dependent graphical models with categorical data in the form of contingency tables. Based on such a decomposition, a covariate-dependent graphical model can be split into some sub-models, and the maximum likelihood estimation of this model can be factorized into the maximum likelihood estimations of the sub-models. Moreover, some sufficient and necessary conditions of the proposed definitions of decomposition are studied.     

Information Bounds and Efficient Estimation in a Class of Censored Transformation Models

Transformation models provide a popular tool for regression analysis of censored failure time data. The most common approach towards parameter estimation in these models is based on nonparametric profile likelihood method. Several authors proposed also ad hoc M-estimators of the Euclidean component of the model. These estimators are usually simpler to implement and many of them have good practical performance. In this paper we consider the form of the information bound for estimation of the Euclidean parameter of the model and propose a modification of the inefficient M-estimators to one-step maximum likelihood estimates.     

Composite likelihood and maximum likelihood methods for joint latent class modeling of disease prevalence and high-dimensional semicontinuous biomarker data

Joint latent class modeling of disease prevalence and high-dimensional semicontinuous biomarker data has been proposed to study the relationship between diseases and their related biomarkers. However, statistical inference of the joint latent class modeling approach has proved very challenging due to its computational complexity in seeking maximum likelihood estimates. In this article, we propose a series of composite likelihoods for maximum composite likelihood estimation, as well as an enhanced Monte Carlo expectation–maximization (MCEM) algorithm for maximum likelihood estimation, in the context of joint latent class models. Theoretically, the maximum composite likelihood estimates are consistent and asymptotically normal. Numerically, we have shown that, as compared to the MCEM algorithm that maximizes the full likelihood, not only the composite likelihood approach that is coupled with the quasi-Newton method can substantially reduce the computational complexity and duration, but it can simultaneously retain comparative estimation efficiency.     

Parameter estimation and model selection for mixtures of truncated exponentials

Bayesian networks with mixtures of truncated exponentials (MTEs) support efficient inference algorithms and provide a flexible way of modeling hybrid domains (domains containing both discrete and continuous variables). On the other hand, estimating an MTE from data has turned out to be a difficult task, and most prevalent learning methods treat parameter estimation as a regression problem. The drawback of this approach is that by not directly attempting to find the parameter estimates that maximize the likelihood, there is no principled way of performing subsequent model selection using those parameter estimates. In this paper we describe an estimation method that directly aims at learning the parameters of an MTE potential following a maximum likelihood approach. Empirical results demonstrate that the proposed method yields significantly better likelihood results than existing regression-based methods. We also show how model selection, which in the case of univariate MTEs amounts to partitioning the domain and selecting the number of exponential terms, can be performed using the BIC score.     

Relational models for contingency tables

The paper considers general multiplicative models for complete and incomplete contingency tables that generalize log-linear and several other models and are entirely coordinate free. Sufficient conditions for the existence of maximum likelihood estimates under these models are given, and it is shown that the usual equivalence between multinomial and Poisson likelihoods holds if and only if an overall effect is present in the model. If such an effect is not assumed, the model becomes a curved exponential family and a related mixed parameterization is given that relies on non-homogeneous odds ratios. Several examples are presented to illustrate the properties and use of such models.     

Nonparametric Bayesian Modeling for Multivariate Ordinal Data

This article proposes a probability model for k-dimensional ordinal outcomes, that is, it considers inference for data recorded in k-dimensional contingency tables with ordinal factors. The proposed approach is based on full posterior inference, assuming a flexible underlying prior probability model for the contingency table cell probabilities. We use a variation of the traditional multivariate probit model, with latent scores that determine the observed data. In our model, a mixture of normals prior replaces the usual single multivariate normal model for the latent variables. By augmenting the prior model to a mixture of normals we generalize inference in two important ways. First, we allow for varying local dependence structure across the contingency table. Second, inference in ordinal multivariate probit models is plagued by problems related to the choice and resampling of cutoffs defined for these latent variables. We show how the proposed mixture model approach entirely removes these problems. We illustrate the methodology with two examples, one simulated dataset and one dataset of interrater agreement.     

Hierarchical subspace models for contingency tables

For the statistical analysis of multiway contingency tables, we propose modeling interaction terms in each maximal compact component of a hierarchical model. By this approach we can search for parsimonious models with smaller degrees of freedom than the usual hierarchical model, while preserving the localization property of the inference in the hierarchical model. This approach also enables us to evaluate the localization property of a given log-affine model. We discuss estimation and exact tests of the proposed model and illustrate the advantage of the proposed modeling with some data sets.     

Multinomial logit models with implicit variable selection

The multinomial logit model is the most widely used model for the unordered multi-category responses. However, applications are typically restricted to the use of few predictors because in the high-dimensional case maximum likelihood estimates frequently do not exist. In this paper we are developing a boosting technique called multinomBoost that performs variable selection and fits the multinomial logit model also when predictors are high-dimensional. Since in multi-category models the effect of one predictor variable is represented by several parameters one has to distinguish between variable selection and parameter selection. A special feature of the approach is that, in contrast to existing approaches, it selects variables not parameters. The method can also distinguish between mandatory predictors and optional predictors. Moreover, it adapts to metric, binary, nominal and ordinal predictors. Regularization within the algorithm allows to include nominal and ordinal variables which have many categories. In the case of ordinal predictors the order information is used. The performance of boosting technique with respect to mean squared error, prediction error and the identification of relevant variables is investigated in a simulation study. The method is applied to the national Indonesia contraceptive prevalence survey and the identification of glass. Results are also compared with the Lasso approach which selects parameters.     

����������������������ŷ���ģ�͵��Ʋ�ǰ���ӯ��ֲ�

本文综述混合效应模型参数估计方面的若干新进展. 平衡混合效应方差分析模型的协方差阵具有一定结构. 对这类模型, 文献[1]提出了参数估计的一种新方法, 称为谱分解法. 新方法的突出特点是, 能同时给出固定效应和方差分量的估计, 前者是线性的, 后者是二次的,且相互独立. 而后, 文献[2--9]证明了谱分解估计的进一步的统计性质, 同时给出了协方差阵对应的估计, 它不仅是正定阵, 而且可获得它的风险函数, 这些文献还研究了谱分解估计与方差分析估计, 极大似然估计, 限制极大似然估计以及最小范数二次无偏估计的关系. 本文综述这一方向的部分研究成果, 并提出一些待进一步研究的问题.     

Generalized least squares and maximum likelihood estimations of multivariate polychoric correlations

This paper investigates the generalized least squares estimation and the maximum likelihood estimation of the parameters in a multivariate polychoric correlations model, based on data from a multidimensional contingency table. Asymptotic properties of the estimators are discussed. An iterative procedure based on the Gauss-Newton algorithm is implemented to produce the generalized least squares estimates and the standard errors estimates. It is shown that via an iteratively reweighted method, the algorithm produces the maximum likelihood estimates as well. Numerical results on the finite sample behaviors of the methods are reported.     

Parameter estimation in general state-space models using particle methods

Particle filtering techniques are a set of powerful and versatile simulation-based methods to perform optimal state estimation in nonlinear non-Gaussian state-space models. If the model includes fixed parameters, a standard technique to perform parameter estimation consists of extending the state with the parameter to transform the problem into an optimal filtering problem. However, this approach requires the use of special particle filtering techniques which suffer from several drawbacks. We consider here an alternative approach combining particle filtering and gradient algorithms to perform batch and recursive maximum likelihood parameter estimation. An original particle method is presented to implement these approaches and their efficiency is assessed through simulation.     

Asymptotic efficiency of the two-stage estimation method for copula-based models

For multivariate copula-based models for which maximum likelihood is computationally difficult, a two-stage estimation procedure has been proposed previously; the first stage involves maximum likelihood from univariate margins, and the second stage involves maximum likelihood of the dependence parameters with the univariate parameters held fixed from the first stage. Using the theory of inference functions, a partitioned matrix in a form amenable to analysis is obtained for the asymptotic covariance matrix of the two-stage estimator. The asymptotic relative efficiency of the two-stage estimation procedure compared with maximum likelihood estimation is studied. Analysis of the limiting cases of the independence copula and Fréchet upper bound help to determine common patterns in the efficiency as the dependence in the model increases. For the Fréchet upper bound, the two-stage estimation procedure can sometimes be equivalent to maximum likelihood estimation for the univariate parameters. Numerical results are shown for some models, including multivariate ordinal probit and bivariate extreme value distributions, to indicate the typical level of asymptotic efficiency for discrete and continuous data.     

Efficient Algorithm for Computing Maximum Likelihood Estimates in Linear Transformation Models

Linear transformation models, which have been extensively studied in survival analysis, include the two special cases: the proportional hazards model and the proportional odds model. Nonparametric maximum likelihood estimation is usually used to derive the efficient estimators. However, due to the large number of nuisance parameters, calculation of the nonparametric maximum likelihood estimator is difficult in practice, except for the proportional hazards model. We propose an efficient algorithm for computing the maximum likelihood estimates, where the dimensionality of the parameter space is dramatically reduced so that only a finite number of equations need to be solved. Moreover, the asymptotic variance is automatically estimated in the computing procedure. Extensive simulation studies indicate that the proposed algorithm works very well for linear transformation models. A real example is presented for an illustration of the new methodology.     

Linear ordinal quasi-symmetry model and decomposition of symmetry for multi-way tables

For multi-way tables with same categories, Bhapkar and Darroch [4] and Agresti [2], p. 440, considered the general order quasi-symmetry model and the ordinal quasi-symmetry model, respectively. Yamamoto, Iwashita and Tomizawa [16] considered the three kinds of linear ordinal quasi-symmetry models, which are special cases of the first and second order quasi-symmetry models. For multi-way tables with same ordinal categories, the present paper proposes the general linear ordinal quasi-symmetry models, which are special cases of the general order quasi-symmetry model, and gives new decomposition of the symmetry model. Moreover it is shown that the likelihood ratio statistic for testing goodness-of-fit of the symmetry model is asymptotically equivalent to the sum of those for testing goodness-of-fit of the decomposed models.     

Practical Extreme Value Modelling of Hydrological Floods and Droughts: A Case Study

Estimation of flood and drought frequencies is important for reservoir design and management, river pollution, ecology and drinking water supply. Through an example based on daily streamflow observations, we introduce a stepwise procedure for estimating quantiles of the hydrological extremes floods and droughts. We fit the generalised extreme value (GEV) distribution by the method of block maxima and the generalised Pareto (GP) distribution by applying the peak over threshold method. Maximum likelihood, penalized maximum likelihood and probability weighted moments are used for parameter estimation. We incorporate trends and seasonal variation in the models instead of splitting the data, and investigate how the observed number of extreme events, the chosen statistical model, and the parameter estimation method effect parameter estimates and quantiles. We find that a seasonal variation should be included in the GEV distribution fitting for floods using block sizes less than one year. When modelling droughts, block sizes of one year or less are not recommended as significant model bias becomes visible. We conclude that the different characteristics of floods and droughts influence the choices made in the extreme value modelling within a common inferential strategy.This revised version was published online in March 2005 with corrections to the cover date.     

������ɷ��Ԫ��̬�ֲ������ļ�����Ȼ����

??Hidden Markov model is widely used in statistical modeling of time, space and state transition data. The definition of hidden Markov multivariate normal distribution is given. The principle of using cluster analysis to determine the hidden state of observed variables is introduced. The maximum likelihood estimator of the unknown parameters in the model is derived. The simulated observation data set is used to test the estimation effect and stability of the method. The characteristic is simple classical statistical inference such as cluster analysis and maximum likelihood estimation. The method solves the parameter estimation problem of complex statistical models.