Computation of reference Bayesian inference for variance components in longitudinal studies

Generalized linear mixed models (GLMMs) have been applied widely in the analysis of longitudinal data. This model confers two important advantages, namely, the flexibility to include random effects and the ability to make inference about complex covariances. In practice, however, the inference of variance components can be a difficult task due to the complexity of the model itself and the dimensionality of the covariance matrix of random effects. Here we first discuss for GLMMs the relation between Bayesian posterior estimates and penalized quasi-likelihood (PQL) estimates, based on the generalization of Harville’s result for general linear models. Next, we perform fully Bayesian analyses for the random covariance matrix using three different reference priors, two with Jeffreys’ priors derived from approximate likelihoods and one with the approximate uniform shrinkage prior. Computations are carried out via the combination of asymptotic approximations and Markov chain Monte Carlo methods. Under the criterion of the squared Euclidean norm, we compare the performances of Bayesian estimates of variance components with that of PQL estimates when the responses are non-normal, and with that of the restricted maximum likelihood (REML) estimates when data are assumed normal. Three applications and simulations of binary, normal, and count responses with multiple random effects and of small sample sizes are illustrated. The analyses examine the differences in estimation performance when the covariance structure is complex, and demonstrate the equivalence between PQL and the posterior modes when the former can be derived. The results also show that the Bayesian approach, particularly under the approximate Jeffreys’ priors, outperforms other procedures.     

Bayesian networks with a logistic regression model for the conditional probabilities

Logistic regression techniques can be used to restrict the conditional probabilities of a Bayesian network for discrete variables. More specifically, each variable of the network can be modeled through a logistic regression model, in which the parents of the variable define the covariates. When all main effects and interactions between the parent variables are incorporated as covariates, the conditional probabilities are estimated without restrictions, as in a traditional Bayesian network. By incorporating interaction terms up to a specific order only, the number of parameters can be drastically reduced. Furthermore, ordered logistic regression can be used when the categories of a variable are ordered, resulting in even more parsimonious models. Parameters are estimated by a modified junction tree algorithm. The approach is illustrated with the Alarm network.     

An optimization-based approach for the design of Bayesian networks

Bayesian networks model conditional dependencies among the domain variables, and provide a way to deduce their interrelationships as well as a method for the classification of new instances. One of the most challenging problems in using Bayesian networks, in the absence of a domain expert who can dictate the model, is inducing the structure of the network from a large, multivariate data set. We propose a new methodology for the design of the structure of a Bayesian network based on concepts of graph theory and nonlinear integer optimization techniques.     

Bayesian parameter inference for models of the Black and Scholes type

In this paper, we describe a general method for constructing the posterior distribution of the mean and volatility of the return of an asset satisfying dS=SdX for some simple models of X. Our framework takes as inputs the prior distributions of the parameters of the stochastic process followed by the underlying, as well as the likelihood function implied by the observed price history for the underlying. As an application of our framework, we compute the value at risk (VaR) and conditional VaR (CVaR) measures for the changes in the price of an option implied by the posterior distribution of the volatility of the underlying. The implied VaR and CVaR are more conservative than their classical counterpart, since it takes into account the estimation risk that arises due to parameter uncertainty. Copyright © 2008 John Wiley & Sons, Ltd.     

Minimax estimators that shift towards a hypersphere for location vectors of spherically symmetric distributions

Let X be a p-dimensional random vector with density f(6X?θ6) where θ is an unknown location vector. For p ≥ 3, conditions on f are given for which there exist minimax estimators θ?(X) satisfying 6Xt6 · 6θ?(X) ? X6 ≤ C, where C is a known constant depending on f. (The positive part estimator is among them.) The loss function is a nondecreasing concave function of 6θ?? θ6². If θ is assumed likely to lie in a ball in $\text{R}$^p, then minimax estimators are given which shrink from the observation X outside the ball in the direction of P(X) the closest point on the surface of the ball. The amount of shrinkage depends on the distance of X from the ball.     

Minimax estimators for location vectors in elliptical distributions with unknown scale parameter and its application to variance reduction in simulation

In this paper, we give an ever wider and new class of minimax estimators for the location vector of an elliptical distribution (a scale mixture of normal densities) with an unknown scale parameter. The its application to variance reduction for Monte Carlo simulation when control variates are used is considered. The results obtained thus extend (i) Berger's result concerning minimax estimation of location vectors for scale mixtures of normal densities with known scale parameter and (ii) Strawderman's result on the estimation of the normal mean with common unknown variance.Research partially supported by National Science Foundation, Grant #DMS 8901922.     

The normal score transformation applied to a multi-univariate method of global optimization

Nonparametric global optimization methods have been developed that determine the location of their next guess based on the rank-transformed objective function evaluations rather than the actual function values themselves. Another commonly-used transformation in nonparametric statistics is the normal score transformation. This paper applies the normal score transformation to the multi-univariate method of global optimization. The benefits of the new method are shown by its performance on a standard set of global optimization test problems. The normal score transformation yields a method that gives equivalent searches for any monotonic transformation of the objective function.     

Bayesian methods in global optimization

This paper reviews methods which have been proposed for solving global optimization problems in the framework of the Bayesian paradigm.     

Paradoxes in conditional probability

It is shown that paradoxes arise in conditional probability calculations, due to incomplete specification of the problem at hand. This is illustrated with the Borel and the Kac-Slepian type paradoxes. These are significant in applications including Bayesian inference. Also Rényi's axiomatic setup does not resolve them. An open problem on calculation of conditional probabilities in the continuous case is noted.     

A survey on the global optimization problem: General theory and computational approaches

Several different approaches have been suggested for the numerical solution of the global optimization problem: space covering methods, trajectory methods, random sampling, random search and methods based on a stochastic model of the objective function are considered in this paper and their relative computational effectiveness is discussed. A closer analysis is performed of random sampling methods along with cluster analysis of sampled data and of Bayesian nonparametric stopping rules.