On the Bayesian Mixture Model and Identifiability

This article is concerned with Bayesian mixture models and identifiability issues. There are two sources of unidentifiability: the well-known likelihood invariance under label switching and the perhaps less well-known parameter identifiability problem. When using latent allocation variables determined by the mixture model, these sources of unidentifiability create arbitrary labeling that renders estimation of the model very difficult. We endeavor to tackle these problems by proposing a prior distribution on the allocations, which provides an explicit interpretation for the labeling by removing gaps with high probability. We propose a Markov chain Monte Carlo (MCMC) estimation method and present supporting illustrations.     

Identifiability analysis of an epidemiological model in a structured population

We investigate the parameter identifiability problem for a system of nonlinear integro-partial differential equations of transport type, representing the spread of a disease with a long infectious but undetectable period in an individual population. After obtaining the expression of the model input-output relationships, we give sufficient conditions on the initial and boundary conditions of the system that guarantee the parameter identifiability on a finite time horizon. We finally illustrate our findings with numerical simulations.     

Bayesian Methods for Wavelet Series in Single-Index Models

Single-index models have found applications in econometrics and biometrics, where multidimensional regression models are often encountered. This article proposes a nonparametric estimation approach that combines wavelet methods for nonequispaced designs with Bayesian models. We consider a wavelet series expansion of the unknown regression function and set prior distributions for the wavelet coefficients and the other model parameters. To ensure model identifiability, the direction parameter is represented via its polar coordinates. We employ ad hoc hierarchical mixture priors that perform shrinkage on wavelet coefficients and use Markov chain Monte Carlo methods for a posteriori inference. We investigate an independence-type Metropolis-Hastings algorithm to produce samples for the direction parameter. Our method leads to simultaneous estimates of the link function and of the index parameters. We present results on both simulated and real data, where we look at comparisons with other methods.     

Asymptotics for estimation and testing procedures under loss of identifiability

Statistical analyses commonly make use of models that suffer from loss of identifiability. In this paper, we address important issues related to the parameter estimation and hypothesis testing in models with loss of identifiability. That is, there are multiple parameter points corresponding to the same true model. We refer the set of these parameter points to as the set of true parameter values. We consider the case where the set of true parameter values is allowed to be very large or even infinite, some parameter values may lie on the boundary of the parameter space, and the data are not necessarily independently and identically distributed. Our results are applicable to a large class of estimators and their related testing statistics derived from optimizing an objective function such as a likelihood. We examine three specific examples: (i) a finite mixture logistic regression model; (ii) stationary ARMA processes; (iii) general quadratic approximation using Hellinger distance. The applications to these examples demonstrate the applicability of our results in a broad range of difficult statistical problems.     

Binomial proportion estimation in longitudinal data with non-ignorable non-response

Non-random missing data poses serious problems in longitudinal studies. The binomial distribution parameter becomes to be unidentifiable without any other auxiliary information or assumption when it suffers from ignorable missing data. Existing methods are mostly based on the log-linear regression model. In this article, a model is proposed for longitudinal data with non-ignorable non-response. It is considered to use the pre-test baseline data to improve the identifiability of the post-test parameter. Furthermore, we derive the identified estimation (IE), the maximum likelihood estimation (MLE) and its associated variance for the post-test parameter. The simulation study based on the model of this paper shows that the proposed approach gives promising results.     

Computation of failure probabilities via local approximations

For failure probability estimates of large structural systems, the numerical expensive evaluations of the limit state function have to be replaced by suitable approximations. Most of the methods proposed in the literature so far construct global approximations of the failure hypersurface. The global approximation of the failure hypersurface does not correspond to the local character of the most likely failure, which is often concentrated in one or several regions in the design space, and may therefore introduce a high approximation error for the probability of failure. Moreover, it is noted that global approximations are often constructed for parameter spaces that ignore constraints imposed by the physical nature of the problem. (© 2006 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Estimation Stability With Cross-Validation (ESCV)

Cross-validation (CV) is often used to select the regularization parameter in high-dimensional problems. However, when applied to the sparse modeling method Lasso, CV leads to models that are unstable in high-dimensions, and consequently not suited for reliable interpretation. In this article, we propose a model-free criterion ESCV based on a new estimation stability (ES) metric and CV. Our proposed ESCV finds a smaller and locally ES-optimal model smaller than the CV choice so that it fits the data and also enjoys estimation stability property. We demonstrate that ESCV is an effective alternative to CV at a similar easily parallelizable computational cost. In particular, we compare the two approaches with respect to several performance measures when applied to the Lasso on both simulated and real datasets. For dependent predictors common in practice, our main finding is that ESCV cuts down false positive rates often by a large margin, while sacrificing little of true positive rates. ESCV usually outperforms CV in terms of parameter estimation while giving similar performance as CV in terms of prediction. For the two real datasets from neuroscience and cell biology, the models found by ESCV are less than half of the model sizes by CV, but preserves CV's predictive performance and corroborates with subject knowledge and independent work. We also discuss some regularization parameter alignment issues that come up in both approaches. Supplementary materials are available online.     

Non-linear model parameter estimation–estimating a feasible parameter set with respect to model use

This article deals with non-linear model parameter estimation from experimental data. As for non-linear models a rigorous identifiability analysis is difficult to perform, parameter estimation is performed in such a way that uncertainty in the estimated parameter values is represented by the range of model use results when the model is used for a certain purpose. Using this approach, the article presents a simulation study where the objective is to discover whether the estimation of model parameters can be improved, so that a small enough range of model use results is obtained. The results of the study indicate that from plant measurements available for the estimation of model parameters, it is possible to extract data that are important for the estimation of model parameters relative to a certain model use. If these data are improved by a proper measurement campaign (e.g. proper choice of measured variables, better accuracy, higher measurement frequency) it is to be expected that a valid model for a certain model use will be obtained. The simulation study is performed for an activated sludge model from wastewater treatment, while the estimation of model parameters is done by Monte Carlo simulation.     

Consistent parameter estimation for partially observed diffusions with small noise

In this paper we provide a consistency result for the MLE for partially observed diffusion processes with small noise intensities. We prove that if the underlying deterministic system enjoys an identifiability property, then any MLE is close to the true parameter if the noise intensities are small enough. The proof uses large deviations limits obtained by PDE vanishing viscosity methods. A deterministic method of parameter estimation is formulated. We also specialize our results to a binary detection problem, and compare deterministic and stochastic notions of identifiability.This research was supported: by Systems Research Center, University of Maryland through NSF Grant CDR-85-00108 and AFOSR-URI Grant 87-0073; by Lefschetz Center for Dynamical Systems, Division of Applied Mathematics, Brown University, under ARO/MIT Grant DAAL-03-86-K-0171; by INRIA Sophia Antipolis, under ERO/INRIA Grant DAJA45-90-C-0008, and by the CNRS-GRAutomatique.     

Identifiability issues of age–period and age–period–cohort models of the Lee–Carter type

The predominant way of modelling mortality rates is the Lee–Carter model and its many extensions. The Lee–Carter model and its many extensions use a latent process to forecast. These models are estimated using a two-step procedure that causes an inconsistent view on the latent variable. This paper considers identifiability issues of these models from a perspective that acknowledges the latent variable as a stochastic process from the beginning. We call this perspective the plug-in age–period or plug-in age–period–cohort model. Defining a parameter vector that includes the underlying parameters of this process rather than its realizations, we investigate whether the expected values and covariances of the plug-in Lee–Carter models are identifiable. It will be seen, for example, that even if in both steps of the estimation procedure we have identifiability in a certain sense it does not necessarily carry over to the plug-in models.     

Computation of an infinite integral using series acceleration

A numerical computation in crystallography involves an infinite integral depending on one parameter. In a recent article in this journal this computational problem is addressed using Romberg’s method and tools for error control. One observe numerical difficulties with the reported approach both near the parameter’s endpoints and near the parameter interval’s midpoint. In this short note we will present an alternative approach making use of a known infinite series formulation of the problem at hand and a simple and efficient series acceleration technique. If some care is taken to avoid cancellations the numerical results are excellent for all values of the parameter. AMS subject classification 65B05, 65B10, 65D30     

Selection of parameter values in environmental models using sparse data: A case study

Models of environmental processes must often be constructed without the use of extensive data sets. This can occur because the exercise is preliminary (aimed at guiding future data collection) or because requisite data are extremely difficult, expensive, or even impossible to obtain. In such cases traditional, statistically based methods for estimating parameters in the model cannot be applied; in fact, parameter estimation cannot be accomplished in a rigorous way at all. We examine the use of a regionalized sensitivity analysis procedure to select appropriate values for parameters in cases where only sparse, imprecise data are available. The utility of the method is examined in the context of equilibrium and dynamic models for describing water quality and hydrological data in a small catchment in Shehandoah National Park, Virginia. Results demonstrate that (1) models can be “tentatively calibrated” using this procedure; (2) the data most likely to provide a stringent test of the model can be identified; and (3) potential problems with model identifiability can be exposed in a preliminary analysis.     

Local parameter identifiability for one class of hybrid systems

We consider the problem of local parameter identifiability for a hybrid system with components that are continuous and discrete in time. The set of observations is the vector-solution (depending on the parameter that is continuous in time) of the discrete component. Sufficient conditions of local parameter identifiability have been formulated using the earlier introduced notion of normalized separability of the set of parameters from the kernel of a special functional. An example where the condition of normed separability is reduced to some rank criterion is given.     

On the identifiability of measurement error in the bifurcating autoregressive model

Huggins and Staudte (1994) considered a mixed linear model for the analysis of cell lineage data and in models for the covariance structure which involved measurement error, it was not immediately clear that the parameters involved were identifiable. Whilst a numerical examination of the Hessian matrix at the estimated parameter values gave some reassurance, this was not theoretically satisfying. Here a matrix formulation of the robust estimating functions of Huggins (1993a, b) as applied in Huggins and Staudte (1994), which include the maximum likelihood estimating functions under the assumption of multivariate normality as a special case, is given along with a direct proof linking identifiability expressed in terms of the estimating functions with the information matrix or its analogue in more general settings. The resulting conditions on the estimating functions may then be checked globally using computer algebra, suggesting a method for establishing identifiability in mixed linear models in general.     

Taylor方法在CSTR河流水质模型结构可识别性分析中的应用

参数识别是水质模型应用的重要环节,结构可识别性是水质模型参数可识别性的基础.采用Taylor方法,对单河段CSTR模型的结构可识别性进行了研究.结果表明,以单河段首端水质作为输入,以末端水质作为输出,考虑COD_(Mn)、NH_3-N、NO_3-N和DO四个水质变量,CSTR模型在结构上是可以识别的.     

Fitting Social Network Models Using Varying Truncation Stochastic Approximation MCMC Algorithm

The exponential random graph model (ERGM) plays a major role in social network analysis. However, parameter estimation for the ERGM is a hard problem due to the intractability of its normalizing constant and the model degeneracy. The existing algorithms, such as Monte Carlo maximum likelihood estimation (MCMLE) and stochastic approximation, often fail for this problem in the presence of model degeneracy. In this article, we introduce the varying truncation stochastic approximation Markov chain Monte Carlo (SAMCMC) algorithm to tackle this problem. The varying truncation mechanism enables the algorithm to choose an appropriate starting point and an appropriate gain factor sequence, and thus to produce a reasonable parameter estimate for the ERGM even in the presence of model degeneracy. The numerical results indicate that the varying truncation SAMCMC algorithm can significantly outperform the MCMLE and stochastic approximation algorithms: for degenerate ERGMs, MCMLE and stochastic approximation often fail to produce any reasonable parameter estimates, while SAMCMC can do; for nondegenerate ERGMs, SAMCMC can work as well as or better than MCMLE and stochastic approximation. The data and source codes used for this article are available online as supplementary materials.     

复合Linex对称损失下Kumaraswamy分布参数的Bayes估计

当参数的先验分布为伽玛分布时,在复合Linex对称损失函数下得到了Kumaraswamy分布参数θ的唯一的Bayes估计,多层Bayes估计和E-Bayes估计,并通过数值模拟说明了所给参数估计的稳健性和精确性.     

A Group-Theoretic Approach to Parameter Identifiability of PDE Systems

The contribution is devoted to the parameter identifiability problem of (nonlinear) PDE systems. Especially, we discuss the (local) identifiability of parameters along a trajectory. The analysis relies on a coordinate-free formulation for systems, including boundary conditions, and we motivate an approach by (Lie) transformation groups, whose success for PDE systems depends on a consequent extent to the accompanying boundary conditions. The (non-)identifiability of parameters is related to the (non-)existence of group generators, wherewith (local) conditions can be derived. (© 2010 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Numerical Computation of Multivariate Normal Probabilities

Abstract

The numerical computation of a multivariate normal probability is often a difficult problem. This article describes a transformation that simplifies the problem and places it into a form that allows efficient calculation using standard numerical multiple integration algorithms. Test results are presented that compare implementations of two algorithms that use the transformation with currently available software.     

双边定时截尾样本下广义逆指数分布形状参数的估计

先给出了广义逆指数分布在双边定时截尾样本下形状参数的最大似然估计,并不能得到估计的显式表达式,但证明了参数在(0,+∞)上最大似然估计是唯一存在的.其次提出用EM算法求出形状参数的估计且该估计具有良好的收敛性,还给出了形状参数的EM估计的渐近方差和近似置信区间;最后通过数值模拟,对形状参数的最大似然估计和EM估计的效果进行了比较,说明了用EM算法求形状参数的估计是可行的,并且模拟效果相对比较好.