Mixture Modeling for Longitudinal Data

In this article, we propose an unbiased estimating equation approach for a two-component mixture model with correlated response data. We adapt the mixture-of-experts model and a generalized linear model for component distribution and mixing proportion, respectively. The new approach only requires marginal distributions of both component densities and latent variables. We use serial correlations from subjects’ subgroup memberships, which improves estimation efficiency and classification accuracy, and show that estimation consistency does not depend on the choice of the working correlation matrix. The proposed estimating equation is solved by an expectation-estimating-equation (EEE) algorithm. In the E-step of the EEE algorithm, we propose a joint imputation based on the conditional linear property for the multivariate Bernoulli distribution. In addition, we establish asymptotic properties for the proposed estimators and the convergence property using the EEE algorithm. Our method is compared to an existing competitive mixture model approach in both simulation studies and an election data application. Supplementary materials for this article are available online.     

Application in Biological Susceptibility Patterns of the Reversible Jump Markov Chain Monte Carlo Sampling

In some biological experiments, it is quite common that laboratory subjects may be different in their patterns of susceptibility to a treatment. We need to determine the different patterns of susceptibility. In this paper we model the number of susceptibility's patterns and the parameters jointly, and base inference about these quantities on their posterior probabilities, making use of reversible jump Markov chain Monte Carlo methods that are capable of jumping between the parameter subspaces corresponding to different numbers of components in the mixture. For convenience, we always assume different patterns of susceptibility have common variances. The paper apply the methodology to the analysis of univariate normal mixtures with different variances. The practical significance of the proposed method is illustrated with a dose-response data set.     

A latent variables approach for clustering mixed binary and continuous variables within a Gaussian mixture model

For clustering objects, we often collect not only continuous variables, but binary attributes as well. This paper proposes a model-based clustering approach with mixed binary and continuous variables where each binary attribute is generated by a latent continuous variable that is dichotomized with a suitable threshold value, and where the scores of the latent variables are estimated from the binary data. In economics, such variables are called utility functions and the assumption is that the binary attributes (the presence or the absence of a public service or utility) are determined by low and high values of these functions. In genetics, the latent response is interpreted as the ??liability?? to develop a qualitative trait or phenotype. The estimated scores of the latent variables, together with the observed continuous ones, allow to use a multivariate Gaussian mixture model for clustering, instead of using a mixture of discrete and continuous distributions. After describing the method, this paper presents the results of both simulated and real-case data and compares the performances of the multivariate Gaussian mixture model and of a mixture of joint multivariate and multinomial distributions. Results show that the former model outperforms the mixture model for variables with different scales, both in terms of classification error rate and reproduction of the clusters means.     

A Bayesian nonparametric model and its application in insurance loss prediction

Predicting insurance losses is an eternal focus of actuarial science in the insurance sector. Due to the existence of complicated features such as skewness, heavy tail, and multi-modality, traditional parametric models are often inadequate to describe the distribution of losses, calling for a mature application of Bayesian methods. In this study we explore a Gaussian mixture model based on Dirichlet process priors. Using three automobile insurance datasets, we employ the probit stick-breaking method to incorporate the effect of covariates into the weight of the mixture component, improve its hierarchical structure, and propose a Bayesian nonparametric model that can identify the unique regression pattern of different samples. Moreover, an advanced updating algorithm of slice sampling is integrated to apply an improved approximation to the infinite mixture model. We compare our framework with four common regression techniques: three generalized linear models and a dependent Dirichlet process ANOVA model. The empirical results show that the proposed framework flexibly characterizes the actual loss distribution in the insurance datasets and demonstrates superior performance in the accuracy of data fitting and extrapolating predictions, thus greatly extending the application of Bayesian methods in the insurance sector.     

Estimation of parameters in latent class models using fuzzy clustering algorithms

A mixture approach to clustering is an important technique in cluster analysis. A mixture of multivariate multinomial distributions is usually used to analyze categorical data with latent class model. The parameter estimation is an important step for a mixture distribution. Described here are four approaches to estimating the parameters of a mixture of multivariate multinomial distributions. The first approach is an extended maximum likelihood (ML) method. The second approach is based on the well-known expectation maximization (EM) algorithm. The third approach is the classification maximum likelihood (CML) algorithm. In this paper, we propose a new approach using the so-called fuzzy class model and then create the fuzzy classification maximum likelihood (FCML) approach for categorical data. The accuracy, robustness and effectiveness of these four types of algorithms for estimating the parameters of multivariate binomial mixtures are compared using real empirical data and samples drawn from the multivariate binomial mixtures of two classes. The results show that the proposed FCML algorithm presents better accuracy, robustness and effectiveness. Overall, the FCML algorithm has the superiority over the ML, EM and CML algorithms. Thus, we recommend FCML as another good tool for estimating the parameters of mixture multivariate multinomial models.     

Consequences of refuge and diffusion in a spatiotemporal predator–prey model

In this investigation, we offer and examine a predator–prey interacting model with prey refuge in proportion to both the species and Beddington–DeAngelis functional response. We first prove the well-posedness of the temporal and spatiotemporal models which are restricted in a positive invariant region. Then for the temporal model, we analyse its temporal dynamics including uniform boundedness, permanence, stability of all feasible non-negative equilibria and show that refugia can induce periodic oscillation via Hopf bifurcation around the unique positive equilibrium; for the spatiotemporal model, we not only investigate its permanence, stability of non-negative constant steady states and Turing instability but also study the existence and non-existence of non-constant positive steady states by Leray–Schauder degree theory. The key observation is that the coefficient of refuge cooperates a significant part in modifying the dynamics of the current system and mediates the population permanence, stability of coexisting equilibrium and even the Turing instability parameter space. Finally, general numerical simulation consequences are given to illustrate the validity of the theoretical results. Through numerical simulations, one observes that the model dynamics shows prey refugia and self-diffusion control spatiotemporal pattern growth to spots, stripe–spot mixtures and stripes reproduction. The outcomes assign that the dynamics of the model with prey refuge is not simple, but rich and complex. Additionally, numerical simulations show that the other model parameters have an important effect on species’ spatially inhomogeneous distribution, which results in the formation of spots pattern, mixture of spots and stripes pattern, mixture of spots, stripes and rings pattern and anti-spot pattern. This may improve the model dynamics of the prey refuge on the reaction–diffusion predator–prey system.     

Bayesian cylindrical data modeling using Abe–Ley mixtures

This paper proposes a Metropolis–Hastings algorithm based on Markov chain Monte Carlo sampling, to estimate the parameters of the Abe–Ley distribution, which is a recently proposed Weibull-Sine-Skewed-von Mises mixture model, for bivariate circular-linear data. Current literature estimates the parameters of these mixture models using the expectation-maximization method, but we will show that this exhibits a few shortcomings for the considered mixture model. First, standard expectation-maximization does not guarantee convergence to a global optimum, because the likelihood is multi-modal, which results from the high dimensionality of the mixture’s likelihood. Second, given that expectation-maximization provides point estimates of the parameters only, the uncertainties of the estimates (e.g., confidence intervals) are not directly available in these methods. Hence, extra calculations are needed to quantify such uncertainty. We propose a Metropolis–Hastings based algorithm that avoids both shortcomings of expectation-maximization. Indeed, Metropolis–Hastings provides an approximation to the complete (posterior) distribution, given that it samples from the joint posterior of the mixture parameters. This facilitates direct inference (e.g., about uncertainty, multi-modality) from the estimation. In developing the algorithm, we tackle various challenges including convergence speed, label switching and selecting the optimum number of mixture components. We then (i) verify the effectiveness of the proposed algorithm on sample datasets with known true parameters, and further (ii) validate our methodology on an environmental dataset (a traditional application domain of Abe–Ley mixtures where measurements are function of direction). Finally, we (iii) demonstrate the usefulness of our approach in an application domain where the circular measurement is periodic in time.     

基于EM算法的混合模型中子总体个数的研究

混合模型已成为数据分析中最流行的技术之一,由于拥有数学模型,它通常比聚类分析中的传统的方法产生的结果更精确,而关键因素是混合模型中子总体个数,它决定了数据分析的最终结果。期望最大化(EM)算法常用在混合模型的参数估计,以及机器学习和聚类领域中的参数估计中,是一种从不完全数据或者是有缺失值的数据中求解参数极大似然估计的迭代算法。学者们往往采用AIC和BIC的方法来确定子总体的个数,而这两种方法在实际的应用中的效果并不稳定,甚至可能会产生错误的结果。针对此问题,本文提出了一种利用似然函数的碎石图来确定混合模型中子总体的个数的新方法。实验结果表明,本文方法确定的子总体的个数在大部分理想的情况下可以得到与AIC、BIC方法确定的聚类个数相同的结果,而在一般的实际数据中或条件不理想的状态下,碎石图方法也可以得到更可靠的结果。随后,本文将新方法在选取的黄石公园喷泉数据的参数估计中进行了实际的应用。     

正态分布和瑞利分布混合情形下的参数估计及分类问题

学者往往用单一的分布模拟和拟合杂波,如正态分布、瑞利分布和威布尔分布等。然而在实际中,雷达杂波由多种类型的杂波组成,单一分布通常不能精确刻画雷达杂波规律,因此,应用混合分布模型对雷达杂波数据建模更准确。本文考虑用正态分布和瑞利分布的混合分布拟合杂波,并应用矩估计方法和基于EM算法的极大似然估计方法估计模型参数,最后,应用最大后验概率分类准则验证2种估计方法的分类准确率。通过数据模拟,得出极大似然估计的效果和分类准确率都要优于矩估计的估计效果和分类准确率。     

A mixture experiments multi-objective hyper-heuristic

This paper proposes a hyper-heuristic that combines genetic algorithm with mixture experiments to solve multi-objective optimisation problems. At every iteration, the proposed algorithm combines the selection criterion (rank indicator) of a number of well-established evolutionary algorithms including NSGA-II, SPEA2 and two versions of IBEA. Each indicator is called according to an associated probability calculated and updated during the search by means of mixture experiments. Mixture experiments are a particular type of experimental design suitable for the calibration of parameters that represent probabilities. Their main output is an explanatory model of algorithm performance as a function of its parameters. By finding the maximum (probability distribution) of the surface represented by this model, we also find a good algorithm parameterisation. The design of mixture experiments approach allowed the authors to identify and exploit synergies between the different rank indicators at the different stages of the search. This is demonstrated by our experimental results in which the proposed algorithm compared favourably against other well-established algorithms.     

Reducing lost-sales rate on the stochastic inventory model with defective goods for the mixtures of distributions

In this paper, we assume that the demands of different customers are not identical in the lead time. Thus, we investigate a continuous review inventory model involving controllable lead time and a random number of defective goods in buyer’s arriving order lot with partial lost sales for the mixtures of distributions of the lead time demand to accommodate more practical features of the real inventory systems. Moreover, we analyze the effects of increasing investment to reduce the lost sales rate when the order quantity, reorder point, lost sales rate and lead time are treated as decision variables. In our studies, we first assume that the lead time demand follows the mixture of normal distributions, and then relax the assumption about the form of the mixture of distribution functions of the lead time demand and apply the minimax distribution free procedure to solve the problem. By analyzing the total expected cost function, we develop an algorithm to obtain the optimal ordering policy and the optimal investment strategy for each case. Finally, we provide numerical examples to illustrate the results.     

利用不一致同胞数据作定量性状的关联检验

为检验定量位点和单个标记位点的等位基因之间的关联性,Schork等[2000]提出了一个odds ratio检验,这个检验是基于抽取不相干“病例”和“对照”个体．但是该检验会因人群混合或分层的影响而失效．受Risch.Zhang[1995]和Schork等[2000]的启发,我们考虑采用非一致同胞数据,并提出一个简单的检验方法．由于该检验是基于家庭的,对人群混合或分层不敏感．该检验很容易推广到多个标记位点的情形,并可以利用所有的含有至少一个“病例”个体和一个“对照”个体的同胞数据．通过数值计算讨论了检验的统计性质,特别研究了高分位点和低分位点(用来定义非一致同胞)的选取对检验功效的影响,讨论的结果对实际应用具有指导作用．进一步的随机模拟表明,利用多个标记位点数据能够明显地提高功效．此外,通过随机模拟比较了文中提出的检验和Allison等[1999]的检验的功效．结果显示,如果适当地选取两个分位点,我们的检验更有效．     

Comparative statics of a signaling game: An experimental study

In this paper a simple and basic signaling game is studied in an experimental environment. First, we check whether we can replicate some of the findings in the literature concerning equilibrium selection and the use and impact of costly signals. Second, and foremost, the comparative statics implications of the game are studied. The experimental results are related to the predictions of two competing behavioral models: a game model, in which subjects are assumed to behave in line with (refined) sequential equilibrium theory, and a decision model, in which subjects are assumed to behave as non-strategic decision makers. The experimental outcomes replicate the finding in the literature that costly messages are sent more frequently by ‘higher’ sender types (whose information is such that persuasion is also profitable to the responder), and that such messages have an impact on the behavior of the responder. These results are consistent with (versions of) both the game model and the decision model. The comparative statics results, however, clearly point in the direction of the decision model. Play is most strongly affected by ‘own’ payoff parameters, as predicted by the decision model, and less so by opponent's payoff parameters, as predicted by the mixed strategies of the refined sequential equilibrium. Particularly, a decision model in which players are assumed to adapt beliefs about opponents' choice probabilities in response to experience in previous play, appears to succeed best in organizing the data.     

线性模型中假设是凸锥时似然比统计量的零分布

对于有正态误差和已知协方差阵的线性模型,讨论了参数域是凸锥的假设检验问题．在考察了似然比统计量的性质后,表明了只要似然比统计量是观察值的凸函数,则似然比统计量的零分布是Ｘ２－分布的混合,而此前的结果是仅当零假设或备择假设形成线性空间时才可用．     

Semiparametric mixtures of regressions with single-index for model based clustering

In this article, we propose two classes of semiparametric mixture regression models with single-index for model based clustering. Unlike many semiparametric/nonparametric mixture regression models that can only be applied to low dimensional predictors, the new semiparametric models can easily incorporate high dimensional predictors into the nonparametric components. The proposed models are very general, and many of the recently proposed semiparametric/nonparametric mixture regression models are indeed special cases of the new models. Backfitting estimates and the corresponding modified EM algorithms are proposed to achieve optimal convergence rates for both parametric and nonparametric parts. We establish the identifiability results of the proposed two models and investigate the asymptotic properties of the proposed estimation procedures. Simulation studies are conducted to demonstrate the finite sample performance of the proposed models. Two real data applications using the new models reveal some interesting findings.

     

Adaptive Rejection Metropolis Simulated Annealing for Detecting Global Maximum Regions

A finite mixture model has been used to fit the data from heterogeneous populations to many applications. An Expectation Maximization (EM) algorithm is the most popular method to estimate parameters in a finite mixture model. A Bayesian approach is another method for fitting a mixture model. However, the EM algorithm often converges to the local maximum regions, and it is sensitive to the choice of starting points. In the Bayesian approach, the Markov Chain Monte Carlo (MCMC) sometimes converges to the local mode and is difficult to move to another mode. Hence, in this paper we propose a new method to improve the limitation of EM algorithm so that the EM can estimate the parameters at the global maximum region and to develop a more effective Bayesian approach so that the MCMC chain moves from one mode to another more easily in the mixture model. Our approach is developed by using both simulated annealing (SA) and adaptive rejection metropolis sampling (ARMS). Although SA is a well-known approach for detecting distinct modes, the limitation of SA is the difficulty in choosing sequences of proper proposal distributions for a target distribution. Since ARMS uses a piecewise linear envelope function for a proposal distribution, we incorporate ARMS into an SA approach so that we can start a more proper proposal distribution and detect separate modes. As a result, we can detect the maximum region and estimate parameters for this global region. We refer to this approach as ARMS annealing. By putting together ARMS annealing with the EM algorithm and with the Bayesian approach, respectively, we have proposed two approaches: an EM-ARMS annealing algorithm and a Bayesian-ARMS annealing approach. We compare our two approaches with traditional EM algorithm alone and Bayesian approach alone using simulation, showing that our two approaches are comparable to each other but perform better than EM algorithm alone and Bayesian approach alone. Our two approaches detect the global maximum region well and estimate the parameters in this region. We demonstrate the advantage of our approaches using an example of the mixture of two Poisson regression models. This mixture model is used to analyze a survey data on the number of charitable donations.     

A thermodynamic framework for a mixture of two liquids

In this study, we extend a thermodynamic framework that has been used with some success for describing the response of a variety of single constituent continua. Using the thermodynamic framework, we obtain a model for the mixture of two compressible fluids that has a much simpler structure than the model obtained earlier within the context of mixture theory. We also investigate the response of a mixture of two fluids that is constrained to have a constant volume, using the same thermodynamic framework.     

Free energy fluctuations for a mixture of directed polymers

We study the free energy fluctuations for a mixture of directed polymers, which was first introduced by Borodin et al. (2015) to investigate the limiting distribution of a stationary Kardar-Parisi-Zhang (KPZ) equation. The main results consist of both the law of large numbers and the asymptotic fluctuation for the free energy as the model size tends to infinity. In particular, we find the explicit values (which may depend on model parameters) of normalizing constants in the fluctuation. This shows that such a mixture model is in the KPZ university class.     

Variational Approximation for Mixtures of Linear Mixed Models

Mixtures of linear mixed models (MLMMs) are useful for clustering grouped data and can be estimated by likelihood maximization through the Expectation–Maximization algorithm. A suitable number of components is then determined conventionally by comparing different mixture models using penalized log-likelihood criteria such as Bayesian information criterion. We propose fitting MLMMs with variational methods, which can perform parameter estimation and model selection simultaneously. We describe a variational approximation for MLMMs where the variational lower bound is in closed form, allowing for fast evaluation and develop a novel variational greedy algorithm for model selection and learning of the mixture components. This approach handles algorithm initialization and returns a plausible number of mixture components automatically. In cases of weak identifiability of certain model parameters, we use hierarchical centering to reparameterize the model and show empirically that there is a gain in efficiency in variational algorithms similar to that in Markov chain Monte Carlo (MCMC) algorithms. Related to this, we prove that the approximate rate of convergence of variational algorithms by Gaussian approximation is equal to that of the corresponding Gibbs sampler, which suggests that reparameterizations can lead to improved convergence in variational algorithms just as in MCMC algorithms. Supplementary materials for the article are available online.