Analysis of definitive screening designs: Screening vs prediction

The use of definitive screening designs (DSDs) has been increasing since their introduction in 2011. These designs are used to screen factors and to make predictions. We assert that the choice of analysis method for these designs depends on the goal of the experiment, screening, or prediction. In this work, we present simulation results to address the explanatory (screening) use and the predictive use of DSDs. To address the predictive ability of DSDs, we use two 5‐factor DSDs and simultaneously run central composite designs case studies on which we will compare several common analysis methods. Overall, we find that for screening purposes, the Dantzig selector using the Bayesian Information Criterion statistic is a good analysis choice; however, when the goal of analysis is prediction forward selection using the Bayesian Information Criterion statistic produces models with a lower mean squared prediction error.     

混合整数线性模型方差分量的Bayes估计

本文研究了混合整数线性模型方差分量在无信息先验分布和有信息先验分布下Bayes估计，给出了混合整数线性模型方差分量无信息和：有信息先验分布下的极大后验估计和最佳Bayes估计。     

Statistical inference methods and applications of outcome-dependent sampling designs under generalized linear models

A cost-effective sampling design is desirable in large cohort studies with a limited budget due to the high cost of measurements of primary exposure variables. The outcome-dependent sampling (ODS) designs enrich the observed sample by oversampling the regions of the underlying population that convey the most information about the exposure-response relationship. The generalized linear models (GLMs) are widely used in many fields, however, much less developments have been done with the GLMs for data from the ODS designs. We study how to fit the GLMs to data obtained by the original ODS design and the two-phase ODS design, respectively. The asymptotic properties of the proposed estimators are derived. A series of simulations are conducted to assess the finite-sample performance of the proposed estimators. Applications to a Wilms tumor study and an air quality study demonstrate the practicability of the proposed methods.     

线性回归模型的Bayes统计推断

Bayes方法虽融合了样本信息和先验信息，但利用的先验信息都是有历史经验和专家估计所得，因此可靠度不高。该文研究了正态线性回归模型：Y=Xβ＋e，e—N（0，σ^2。L），其中σ^2已知，β为未知参数向量，对传统的Bayes方法进行了改进，即把Bayes方法中的后验信息作为改进Bayes的无验信息并融合样本信息进行统计推断，在二次损失函数下得到了β的改进的Bayes估计。由于改进的Bayes方法的先验信息中有样本信息，因此其准确度比传统的Bayes方法准确度更高。     

On the normality a posteriori for exponential distributions,using the Bayesian estimation

Summary The Bayesian estimation problem for the parameter θ of an exponential probability distribution is considered, when it is assumed that θ has a natural conjugate prior density and a loss-function depending on the squared error is used. It is shown that, with probability one, the posterior density of the Bayesian—centered and scaled parameter converges pointwise to the normal probability density. The weak convergence of the posterior distributions to the normal distribution follows directly. Both correct and incorrect models are studied and the asymptotic normality is stated respectively.     

Computation of reference Bayesian inference for variance components in longitudinal studies

Generalized linear mixed models (GLMMs) have been applied widely in the analysis of longitudinal data. This model confers two important advantages, namely, the flexibility to include random effects and the ability to make inference about complex covariances. In practice, however, the inference of variance components can be a difficult task due to the complexity of the model itself and the dimensionality of the covariance matrix of random effects. Here we first discuss for GLMMs the relation between Bayesian posterior estimates and penalized quasi-likelihood (PQL) estimates, based on the generalization of Harville’s result for general linear models. Next, we perform fully Bayesian analyses for the random covariance matrix using three different reference priors, two with Jeffreys’ priors derived from approximate likelihoods and one with the approximate uniform shrinkage prior. Computations are carried out via the combination of asymptotic approximations and Markov chain Monte Carlo methods. Under the criterion of the squared Euclidean norm, we compare the performances of Bayesian estimates of variance components with that of PQL estimates when the responses are non-normal, and with that of the restricted maximum likelihood (REML) estimates when data are assumed normal. Three applications and simulations of binary, normal, and count responses with multiple random effects and of small sample sizes are illustrated. The analyses examine the differences in estimation performance when the covariance structure is complex, and demonstrate the equivalence between PQL and the posterior modes when the former can be derived. The results also show that the Bayesian approach, particularly under the approximate Jeffreys’ priors, outperforms other procedures.     

限制参数空间上的Fiducial推断

给出了在限制参数空间上,利用Fiducial方法求参数的区间估计的一般方法,并且讨论了一些常见的典型问题,结果表明所得的区间估计是合理的．另外,本文还证明了在限制参数空间上,刻度族和位置族中参数的条件Fiducial分布与无信息先验的Bayes 后验分布一致,推广了Lindely的结论．     

Economical experiments: Bayesian efficient experimental design

We propose and implement a Bayesian optimal design procedure. Our procedure takes as its primitives a class of parametric models of strategic behavior, a class of games (experimental designs), and priors on the behavioral parameters. We select the experimental design that maximizes the information from the experiment. We sequentially sample with the given design and models until only one of the models has viable posterior odds. A model which has low posterior odds in a small collection of models will have an even lower posterior odds when compared to a larger class, and hence we can dismiss it. The procedure can be used sequentially by introducing new models and comparing them to the models that survived earlier rounds of experiments. The emphasis is not on running as many experiments as possible, but rather on choosing experimental designs to distinguish between models in the shortest possible time period. We illustrate this procedure with a simple experimental game with one-sided incomplete information.We acknowledge the financial support from NSF grant #SES-9223701 to the California Institute of Technology. We also acknowledge the research assistance of Eugene Grayver who wrote the software for the experiments.     

The usefulness of Bayesian optimal designs for discrete choice experiments

Recently, the use of Bayesian optimal designs for discrete choice experiments, also called stated choice experiments or conjoint choice experiments, has gained much attention, stimulating the development of Bayesian choice design algorithms. Characteristic for the Bayesian design strategy is that it incorporates the available information about people's preferences for various product attributes in the choice design. This is in contrast with the linear design methodology, which is also used in discrete choice design and which depends for any claims of optimality on the unrealistic assumption that people have no preference for any of the attribute levels. Although linear design principles have often been used to construct discrete choice experiments, we show using an extensive case study that the resulting utility‐neutral optimal designs are not competitive with Bayesian optimal designs for estimation purposes. Copyright © 2011 John Wiley & Sons, Ltd.     

An Extension of Generalized Linear Models to Finite Mixture Outcome Distributions

Finite mixture distributions arise in sampling a heterogeneous population. Data drawn from such a population will exhibit extra variability relative to any single subpopulation. Statistical models based on finite mixtures can assist in the analysis of categorical and count outcomes when standard generalized linear models (GLMs) cannot adequately express variability observed in the data. We propose an extension of GLMs where the response follows a finite mixture distribution and the regression of interest is linked to the mixture’s mean. This approach may be preferred over a finite mixture of regressions when the population mean is of interest; here, only one regression must be specified and interpreted in the analysis. A technical challenge is that the mixture’s mean is a composite parameter that does not appear explicitly in the density. The proposed model maintains its link to the regression through a certain random effects structure and is completely likelihood-based. We consider typical GLM cases where means are either real-valued, constrained to be positive, or constrained to be on the unit interval. The resulting model is applied to two example datasets through Bayesian analysis. Supporting the extra variation is seen to improve residual plots and produce widened prediction intervals reflecting the uncertainty. Supplementary materials for this article are available online.     

Bayesian Variable Selection for Median Regression

When the data has heavy tail feature or contains outliers, conventional variable selection methods based on penalized least squares or likelihood functions perform poorly. Based on Bayesian inference method, we study the Bayesian variable selection problem for median linear models. The Bayesian estimation method is proposed by using Bayesian model selection theory and Bayesian estimation method through selecting the Spike and Slab prior for regression coefficients, and the effective posterior Gibbs sampling procedure is also given. Extensive numerical simulations and Boston house price data analysis are used to illustrate the effectiveness of the proposed method.     

Quadrature Methods for Bayesian Optimal Design of Experiments With Nonnormal Prior Distributions

Many optimal experimental designs depend on one or more unknown model parameters. In such cases, it is common to use Bayesian optimal design procedures to seek designs that perform well over an entire prior distribution of the unknown model parameter(s). Generally, Bayesian optimal design procedures are viewed as computationally intensive. This is because they require numerical integration techniques to approximate the Bayesian optimality criterion at hand. The most common numerical integration technique involves pseudo Monte Carlo draws from the prior distribution(s). For a good approximation of the Bayesian optimality criterion, a large number of pseudo Monte Carlo draws is required. This results in long computation times. As an alternative to the pseudo Monte Carlo approach, we propose using computationally efficient Gaussian quadrature techniques. Since, for normal prior distributions, suitable quadrature techniques have already been used in the context of optimal experimental design, we focus on quadrature techniques for nonnormal prior distributions. Such prior distributions are appropriate for variance components, correlation coefficients, and any other parameters that are strictly positive or have upper and lower bounds. In this article, we demonstrate the added value of the quadrature techniques we advocate by means of the Bayesian D-optimality criterion in the context of split-plot experiments, but we want to stress that the techniques can be applied to other optimality criteria and other types of experimental designs as well. Supplementary materials for this article are available online.     

��λ���ع�ı�Ҷ˹����ѡ�񷽷�

??When the data has heavy tail feature or contains outliers, conventional variable selection methods based on penalized least squares or likelihood functions perform poorly. Based on Bayesian inference method, we study the Bayesian variable selection problem for median linear models. The Bayesian estimation method is proposed by using Bayesian model selection theory and Bayesian estimation method through selecting the Spike and Slab prior for regression coefficients, and the effective posterior Gibbs sampling procedure is also given. Extensive numerical simulations and Boston house price data analysis are used to illustrate the effectiveness of the proposed method.     

Local Derivative-Free Approximation of Computationally Expensive Posterior Densities

Bayesian inference using Markov chain Monte Carlo (MCMC) is computationally prohibitive when the posterior density of interest, π, is computationally expensive to evaluate. We develop a derivative-free algorithm GRIMA to accurately approximate π by interpolation over its high-probability density (HPD) region, which is initially unknown. Our local approach reduces the waste of computational budget on approximation of π in the low-probability region, which is inherent in global experimental designs. However, estimation of the HPD region is nontrivial when derivatives of π are not available or are not informative about the shape of the HPD region. Without relying on derivatives, GRIMA iterates (a) sequential knot selection over the estimated HPD region of π to refine the surrogate posterior and (b) re-estimation of the HPD region using an MCMC sample from the updated surrogate density, which is inexpensive to obtain. GRIMA is applicable to approximation of general unnormalized posterior densities. To determine the range of tractable problem dimensions, we conduct simulation experiments on test densities with linear and nonlinear component-wise dependence, skewness, kurtosis and multimodality. Subsequently, we use GRIMA in a case study to calibrate a computationally intensive nonlinear regression model to real data from the Town Brook watershed. Supplemental materials for this article are available online.     

Probability matching priors for predicting a dependent variable with application to regression models

In a Bayesian setup, we consider the problem of predicting a dependent variable given an independent variable and past observations on the two variables. An asymptotic formula for the relevant posterior predictive density is worked out. Considering posterior quantiles and highest predictive density regions, we then characterize priors that ensure approximate frequentist validity of Bayesian prediction in the above setting. Application to regression models is also discussed.     

��I�ͻ�Ͻ�β��ָ��——�������ֲ�����ʧЧģ�͵�ͳ�Ʒ���

In this paper, we investigate a competing risks model based on exponentiated Weibull distribution under Type-I progressively hybrid censoring scheme. To estimate the unknown parameters and reliability function, the maximum likelihood estimators and asymptotic confidence intervals are derived. Since Bayesian posterior density functions cannot be given in closed forms, we adopt Markov chain Monte Carlo method to calculate approximate Bayes estimators and highest posterior density credible intervals. To illustrate the estimation methods, a simulation study is carried out with numerical results. It is concluded that the maximum likelihood estimation and Bayesian estimation can be used for statistical inference in competing risks model under Type-I progressively hybrid censoring scheme.     

Asymptotic theory of dependent Bayesian multiple testing procedures under possible model misspecification

We study asymptotic properties of Bayesian multiple testing procedures and provide sufficient conditions for strong consistency under general dependence structure. We also consider a novel Bayesian multiple testing procedure and associated error measures that coherently accounts for the dependence structure present in the model. We advocate posterior versions of FDR and FNR as appropriate error rates and show that their asymptotic convergence rates are directly associated with the Kullback–Leibler divergence from the true model. The theories hold regardless of the class of postulated models being misspecified. We illustrate our results in a variable selection problem with autoregressive response variables and compare our procedure with some existing methods through simulation studies. Superior performance of the new procedure compared to the others indicates that proper exploitation of the dependence structure by multiple testing methods is indeed important. Moreover, we obtain encouraging results in a maize dataset, where we select influential marker variables.

     

Variational Approximations for Generalized Linear Latent Variable Models

Generalized linear latent variable models (GLLVMs) are a powerful class of models for understanding the relationships among multiple, correlated responses. Estimation, however, presents a major challenge, as the marginal likelihood does not possess a closed form for nonnormal responses. We propose a variational approximation (VA) method for estimating GLLVMs. For the common cases of binary, ordinal, and overdispersed count data, we derive fully closed-form approximations to the marginal log-likelihood function in each case. Compared to other methods such as the expectation-maximization algorithm, estimation using VA is fast and straightforward to implement. Predictions of the latent variables and associated uncertainty estimates are also obtained as part of the estimation process. Simulations show that VA estimation performs similar to or better than some currently available methods, both at predicting the latent variables and estimating their corresponding coefficients. They also show that VA estimation offers dramatic reductions in computation time particularly if the number of correlated responses is large relative to the number of observational units. We apply the variational approach to two datasets, estimating GLLVMs to understanding the patterns of variation in youth gratitude and for constructing ordination plots in bird abundance data. R code for performing VA estimation of GLLVMs is available online. Supplementary materials for this article are available online.     

Case Influence Analysis in Bayesian Inference

Abstract

We demonstrate how case influence analysis, commonly used in regression, can be applied to Bayesian hierarchical models. Draws from the joint posterior distribution of parameters are importance weighted to reflect the effect of deleting each observation in turn; the ensuing changes in the posterior distribution of each parameter are displayed graphically. The procedure is particularly useful when drawing a sample from the posterior distribution requires extensive calculations (as with a Markov Chain Monte Carlo sampler). The structure of hierarchical models, and other models with local dependence, makes the importance weights inexpensive to calculate with little additional programming. Some new alternative weighting schemes are described that extend the range of problems in which reweighting can be used to assess influence. Applications to a growth curve model and a complex hierarchical model for opinion data are described. Our focus on case influence on parameters is complementary to other work that measures influence by distances between posterior or predictive distributions.     

Bayesian analysis of quantile regression for censored dynamic panel data

This paper develops a Bayesian approach to analyzing quantile regression models for censored dynamic panel data. We employ a likelihood-based approach using the asymmetric Laplace error distribution and introduce lagged observed responses into the conditional quantile function. We also deal with the initial conditions problem in dynamic panel data models by introducing correlated random effects into the model. For posterior inference, we propose a Gibbs sampling algorithm based on a location-scale mixture representation of the asymmetric Laplace distribution. It is shown that the mixture representation provides fully tractable conditional posterior densities and considerably simplifies existing estimation procedures for quantile regression models. In addition, we explain how the proposed Gibbs sampler can be utilized for the calculation of marginal likelihood and the modal estimation. Our approach is illustrated with real data on medical expenditures.