MM Algorithms for Some Discrete Multivariate Distributions

The MM (minorization–maximization) principle is a versatile tool for constructing optimization algorithms. Every EM algorithm is an MM algorithm but not vice versa. This article derives MM algorithms for maximum likelihood estimation with discrete multivariate distributions such as the Dirichlet-multinomial and Connor–Mosimann distributions, the Neerchal–Morel distribution, the negative-multinomial distribution, certain distributions on partitions, and zero-truncated and zero-inflated distributions. These MM algorithms increase the likelihood at each iteration and reliably converge to the maximum from well-chosen initial values. Because they involve no matrix inversion, the algorithms are especially pertinent to high-dimensional problems. To illustrate the performance of the MM algorithms, we compare them to Newton’s method on data used to classify handwritten digits.     

Estimation of parameters in latent class models using fuzzy clustering algorithms

A mixture approach to clustering is an important technique in cluster analysis. A mixture of multivariate multinomial distributions is usually used to analyze categorical data with latent class model. The parameter estimation is an important step for a mixture distribution. Described here are four approaches to estimating the parameters of a mixture of multivariate multinomial distributions. The first approach is an extended maximum likelihood (ML) method. The second approach is based on the well-known expectation maximization (EM) algorithm. The third approach is the classification maximum likelihood (CML) algorithm. In this paper, we propose a new approach using the so-called fuzzy class model and then create the fuzzy classification maximum likelihood (FCML) approach for categorical data. The accuracy, robustness and effectiveness of these four types of algorithms for estimating the parameters of multivariate binomial mixtures are compared using real empirical data and samples drawn from the multivariate binomial mixtures of two classes. The results show that the proposed FCML algorithm presents better accuracy, robustness and effectiveness. Overall, the FCML algorithm has the superiority over the ML, EM and CML algorithms. Thus, we recommend FCML as another good tool for estimating the parameters of mixture multivariate multinomial models.     

ʵ֤����ģ�͵ı�Ҷ˹�����������ģ�ͱȽ�

A Bayesian semiparametric procedure for confirmatory factor analysis model is proposed to address the heterogeneity of the multivariate responses. The approach relies on the use of a prior over the space of mixing distributions with finite components. Blocked Gibbs sampler is implemented to cope with the posterior analysis. For model comparison, themeasure and Bayes factor are developed. A generalized weighted Chinese restaurant algorithm is suggested to compute the likelihood of data. Empirical results are presented to illustrate the effectiveness of the methodologies.     

Statistical mechanics of nonlinear nonequilibrium financial markets: Applications to optimized trading

A paradigm of statistical mechanics of financial markets (SMFM) using nonlinear non-equilibrium algorithms, first published in [1], is fit to multivariate financial markets using Adaptive Simulated Annealing (ASA), a global optimization algorithm, to perform maximum likelihood fits of Lagrangians defined by path integrals of multivariate conditional probabilities. Canonical momenta are thereby derived and used as technical indicators in a recursive ASA optimization process to tune trading rules. These trading rules are then used on out-of-sample data, to demonstrate that they can profit from the SMFM model, to illustrate that these markets are likely not efficient.     

A multivariate stochastic model with non-stationary trend component

The purposes of this paper are to introduce a multivariate non-stationary stochastic time series model without individual detrending and to extract the multiple relationships between variables. To infer the statistical relation between variables, we attempt to estimate the co-movement of multivariate non-stationary time series components. The model is expressed in state-space form, and time series components are estimated by the maximum likelihood method using numerical optimization algorithm. The Kalman filter algorithm is used to compute the likelihood of the model. The AIC procedure gives a criterion for selecting the best model fit for the data. The multiple relationship becomes clear by analysing estimated AR coefficients. Real economic data are used for a numerical example.     

Computational Strategies for Multivariate Linear Mixed-Effects Models With Missing Values

This article presents new computational techniques for multivariate longitudinal or clustered data with missing values. Current methodology for linear mixed-effects models can accommodate imbalance or missing data in a single response variable, but it cannot handle missing values in multiple responses or additional covariates. Applying a multivariate extension of a popular linear mixed-effects model, we create multiple imputations of missing values for subsequent analyses by a straightforward and effective Markov chain Monte Carlo procedure. We also derive and implement a new EM algorithm for parameter estimation which converges more rapidly than traditional EM algorithms because it does not treat the random effects as “missing data,” but integrates them out of the likelihood function analytically. These techniques are illustrated on models for adolescent alcohol use in a large school-based prevention trial.     

Maximum likelihood estimation for multivariate skew normal mixture models

This paper provides a flexible mixture modeling framework using the multivariate skew normal distribution. A feasible EM algorithm is developed for finding the maximum likelihood estimates of parameters in this context. A general information-based method for obtaining the asymptotic covariance matrix of the maximum likelihood estimators is also presented. The proposed methodology is illustrated with a real example and results are also compared with those obtained from fitting normal mixtures.     

Multi-sample cluster analysis using Akaike's Information Criterion

Summary Multi-sample cluster analysis, the problem of grouping samples, is studied from an information-theoretic viewpoint via Akaike's Information Criterion (AIC). This criterion combines the maximum value of the likelihood with the number of parameters used in achieving that value. The multi-sample cluster problem is defined, and AIC is developed for this problem. The form of AIC is derived in both the multivariate analysis of variance (MANOVA) model and in the multivariate model with varying mean vectors and variance-covariance matrices. Numerical examples are presented for AIC and another criterion calledw-square. The results demonstrate the utility of AIC in identifying the best clustering alternatives. This research was supported by Office of Naval Research Contract N00014-80-C-0408, Task NR042-443 and Army Research Office Contract DAAG 29-82-K-0155, at the University of Illinois at Chicago.     

RECURSIVE SYSTEM IDENTIFICATION

Most of existing methods in system identification with possible exception of those for linear systems are off-line in nature, and hence are nonrecursive. This paper demonstrates the recent progress in recursive system identification. The recursive identification algorithms are presented not only for linear systems （multivariate ARMAX systems） but also for nonlinear systems such as the Hammerstein and Wiener systems, and the nonlinear ARX systems. The estimates generated by the algorithms are online updated and converge a.s. to the true values as time tends to infinity.     

Additive Positive Stable Frailty Models

In this article, we describe an additive stable frailty model for multivariate times to events data using a flexible baseline hazard, and assuming that the frailty component for each individual is described by additive functions of independent positive stable random variables with possibly different stability indices. Dependence properties of this frailty model are investigated. To carry out inference, the likelihood function is derived by replacing high-dimensional integration by Monte Carlo simulation. Markov chain Monte Carlo algorithms enable estimation and model checking in the Bayesian framework.      

正态条件下带AR(1)-型方差结构GMANOVA-MANOVA模型极大似然估计的小样本特征

研究了一类带一阶自回归(AR(1))-型方差结构的广义多元方差分析-多元方差分析(GMANO VA-MANOVA)模型参数极大似然估计的小样本特征.对带AR(1)-型方差结构GMANOVA-MANOVA模型,文章在正态条件下给出了参数极大似然估计存在的一个充分必要条件,讨论了极大似然估计唯一的充分条件.在该充分条件下,文章证明了相关系数极大似然估计的精确分布只与相关系数有关,并依此给出了自相关系数简单假设H0:ρ=0v.s.H1:ρ≠0的一个不需要叠代计算估计的检验,同时模拟表明该检验为无偏检验且势函数与似然比检验势函数无太大差异.     

Solving the Likelihood Equations

Given a model in algebraic statistics and data, the likelihood function is a rational function on a projective variety. Algebraic algorithms are presented for computing all critical points of this function, with the aim of identifying the local maxima in the probability simplex. Applications include models specified by rank conditions on matrices and the Jukes–Cantor models of phylogenetics. The maximum likelihood degree of a generic complete intersection is also determined.     

Functional relationships having many independent variables and errors with multivariate normal distribution

This paper deals with maximum likelihood estimation of linear or nonlinear functional relationships assuming that replicated observations have been made on p variables at n points. The joint distribution of the pn errors is assumed to be multivariate normal. Existing results are extended in two ways: first, from known to unknown error covariance matrix; second, from the two variate to the multivariate case.For the linear relationship it is shown that the maximum likelihood point estimates are those obtained by the method of generalized least squares. The present method, however, has the advantage of supplying estimates of the asymptotic covariances of the structural parameter estimates.     

Bayesian Nonparametric Inference for the Power Likelihood

The aim in this article is to provide a means to undertake Bayesian inference for mixture models when the likelihood function is raised to a power between 0 and 1. The main purpose for doing this is to guarantee a strongly consistent model and hence, make it possible to compare the consistent posterior with the correct posterior, looking for signs of discrepancy. This will be explained in detail in the article. Another purpose would be for simulated annealing algorithms. In particular, for the widely used mixture of Dirichlet process model, it is far from obvious how to undertake inference via Markov chain Monte Carlo methods when the likelihood is raised to a power other than 1. In this article, we demonstrate how posterior sampling can be carried out when using a power likelihood. Matlab code to implement the algorithm is available as supplementary material.     

Global Optimization using Dynamic Search Trajectories

Two global optimization algorithms are presented. Both algorithms attempt to minimize an unconstrained objective function through the modeling of dynamic search trajectories. The first, namely the Snyman–Fatti algorithm, originated in the 1980's and still appears an effective global optimization algorithm. The second algorithm is currently under development, and is denoted the modified bouncing ball algorithm. For both algorithms, the search trajectories are modified to increase the likelihood of convergence to a low local minimum. Numerical results illustrate the effectiveness of both algorithms.     

Estimation of the arrival times of seismic waves by multivariate time series model

A computationally efficient procedure was developed for the fitting of many multivariate locally stationary autoregressive models. The details of the Householder method for fitting multivariate autoregressive model and multivariate locally stationary autoregressive model (MLSAR model) are shown. The proposed procedure is quite efficient in both accuracy and computation. The amount of computation is bounded by a multiple of Nm ² with N being the data length and m the highest model order, and does not depend on the number of models checked. This facilitates the precise estimation of the change point of the AR model. Based on the AICs' of the fitted MLSAR models and Akaike's definition of the likelihood of the models, a method of evaluating the posterior distribution of the change point of the AR model is also presented. The proposed procedure is, in particular, useful for the estimation of the arrival time of the S wave of a microearthquake. To illustrate the usefulness of the proposed procedure, the seismograms of the foreshocks of the 1982 Urakawa-Oki Earthquake were analyzed. These data sets have been registered to AISM Data Library and the readers of this Journal can access to them by the method described in this issue.A part of this research was carried out under the ISM Cooperative Research Program (89-ISM.CRP-57).Also with the Faculty of Economics, the University of Tokyo. The author was supported in part by the Japanese Ministry of Education, Science and Culture under Grant-in-Aid for Developmental Scientific Research 63830002.     

Normal Linear Regression Models With Recursive Graphical Markov Structure

A multivariate normal statistical model defined by the Markov properties determined by an acyclic digraph admits a recursive factorization of its likelihood function (LF) into the product of conditional LFs, each factor having the form of a classical multivariate linear regression model (≡WMANOVA model). Here these models are extended in a natural way to normal linear regression models whose LFs continue to admit such recursive factorizations, from which maximum likelihood estimators and likelihood ratio (LR) test statistics can be derived by classical linear methods. The central distribution of the LR test statistic for testing one such multivariate normal linear regression model against another is derived, and the relation of these regression models to block-recursive normal linear systems is established. It is shown how a collection of nonnested dependent normal linear regression models (≡Wseemingly unrelated regressions) can be combined into a single multivariate normal linear regression model by imposing a parsimonious set of graphical Markov (≡Wconditional independence) restrictions.     

Maximum likelihood estimation of covariance matrices under simple tree ordering

The closed-form maximum likelihood estimators for the completely balanced multivariate one-way random effect model are obtained by Anderson et al. (Ann. Statist. 14 (1986) 405). It remains open whether there exist the closed-form maximum likelihood estimators for the more general completely balanced multivariate multi-way random effects models. In this paper, a new parameterization technique for covariance matrices is used to grasp the inside structure of likelihood function so that the maximum likelihood equations can be dramatically simplified. As such we obtain the closed-form maximum likelihood estimators of covariance matrices for Wishart density functions over the simple tree ordering set, which can then be applied to get the maximum likelihood estimators for the completely balanced multivariate multi-way random effects models without interactions.     

Fast Decoding Algorithms for First Order Reed-Muller and Related Codes

Fast decoding algorithms for short codes based on modifications of maximum likelihood decoding algorithms of first order Reed-Muller codes are described. Only additions-subtractions, comparisons and absolute value calculations are used in the algorithms. Soft and hard decisions maximum likelihood decoding algorithms for first order Reed-Muller and the Nordstrom-Robinson codes with low complexity are proposed.     

Balanced random interval arithmetic in market model estimation

The possibility of estimating bounds for the econometric likelihood function using balanced random interval arithmetic is experimentally investigated. The experiments on the likelihood function with data from housing starts have proved the assumption that distributions of centres and radii of evaluated balanced random intervals are normal. Balanced random interval arithmetic can therefore be used to estimate bounds for this function and global optimization algorithms based on this arithmetic are applicable to optimize it. The interval branch and bound algorithms with bounds calculated using standard and balanced random interval arithmetic were used to optimize the likelihood function. Results of the experiments show that when reliability is essential the algorithm with standard interval arithmetic should be used, but when speed of optimization is more important, the algorithm with balanced random interval arithmetic should be used which in this case finishes faster and provides good, although not always optimal, values.