A Note on Algebraic Solutions to Identification

Algebraic methods to establish the identification of structural equation models remain a viable option. However, sometimes it is unclear whether the algebraic solution establishes identification. One example is when there is more than one way to solve for the parameter, but one way leads to a single value and a second way leads to a function with more than one value. This article proves that one explicit and unique solution is sufficient for model identification even when other explicit solutions permit more than one solution. The results are illustrated with an example. The results are useful to attempts to use algebraic means to address model identification.     

Nearest-neighbour modelling of reciprocal chains

This paper focuses on the class of finite-state, discrete-index, reciprocal processes (reciprocal chains). Such a class of processes seems to be a suitable setup in many applications and, in particular, it appears well-suited for image-processing. While addressing this issue, the aim is 2-fold: theoretic and practical. As to the theoretic purpose, some new results are provided: first, a general stochastic realization result is provided for reciprocal chains endowed with a known, arbitrary, distribution. Such a model has the form of a fixed-degree, nearest-neighbour polynomial model. Next, the polynomial model is shown to be exactly linearizable, which means it is equivalent to a nearest-neighbour linear model in a different set of variables. The latter model turns out to be formally identical to the Levi–Frezza–Krener linear model of a Gaussian reciprocal process, although actually non-linear with respect to the chain's values. As far as the practical purpose is concerned, in order to yield an example of application an estimation issue is addressed: a suboptimal (polynomial-optimal) solution is derived for the smoothing problem of a reciprocal chain partially observed under non-Gaussian noise. To this purpose, two kinds of boundary conditions (Dirichlet and Cyclic), specifying the reciprocal chain on a finite interval, are considered, and in both cases the model is shown to be well-posed, in a ‘wide-sense’. Under this view, some well-known representation results about Gaussian reciprocal processes extend, in a sense, to a ‘non-Gaussian’ case.     

Case Influence Analysis in Bayesian Inference

Abstract

We demonstrate how case influence analysis, commonly used in regression, can be applied to Bayesian hierarchical models. Draws from the joint posterior distribution of parameters are importance weighted to reflect the effect of deleting each observation in turn; the ensuing changes in the posterior distribution of each parameter are displayed graphically. The procedure is particularly useful when drawing a sample from the posterior distribution requires extensive calculations (as with a Markov Chain Monte Carlo sampler). The structure of hierarchical models, and other models with local dependence, makes the importance weights inexpensive to calculate with little additional programming. Some new alternative weighting schemes are described that extend the range of problems in which reweighting can be used to assess influence. Applications to a growth curve model and a complex hierarchical model for opinion data are described. Our focus on case influence on parameters is complementary to other work that measures influence by distances between posterior or predictive distributions.     

Functional Additive Mixed Models

We propose an extensive framework for additive regression models for correlated functional responses, allowing for multiple partially nested or crossed functional random effects with flexible correlation structures for, for example, spatial, temporal, or longitudinal functional data. Additionally, our framework includes linear and nonlinear effects of functional and scalar covariates that may vary smoothly over the index of the functional response. It accommodates densely or sparsely observed functional responses and predictors which may be observed with additional error and includes both spline-based and functional principal component-based terms. Estimation and inference in this framework is based on standard additive mixed models, allowing us to take advantage of established methods and robust, flexible algorithms. We provide easy-to-use open source software in the pffr() function for the R package refund. Simulations show that the proposed method recovers relevant effects reliably, handles small sample sizes well, and also scales to larger datasets. Applications with spatially and longitudinally observed functional data demonstrate the flexibility in modeling and interpretability of results of our approach.     

An Approximation Method for Improving Dynamic Network Model Fitting

There has been a great deal of interest recently in the modeling and simulation of dynamic networks, that is, networks that change over time. One promising model is the separable temporal exponential-family random graph model (ERGM) of Krivitsky and Handcock, which treats the formation and dissolution of ties in parallel at each time step as independent ERGMs. However, the computational cost of fitting these models can be substantial, particularly for large, sparse networks. Fitting cross-sectional models for observations of a network at a single point in time, while still a nonnegligible computational burden, is much easier. This article examines model fitting when the available data consist of independent measures of cross-sectional network structure and the duration of relationships under the assumption of stationarity. We introduce a simple approximation to the dynamic parameters for sparse networks with relationships of moderate or long duration and show that the approximation method works best in precisely those cases where parameter estimation is most likely to fail—networks with very little change at each time step. We consider a variety of cases: Bernoulli formation and dissolution of ties, independent-tie formation and Bernoulli dissolution, independent-tie formation and dissolution, and dependent-tie formation models.     

Applications of Hybrid Monte Carlo to Bayesian Generalized Linear Models: Quasicomplete Separation and Neural Networks

Abstract

The “leapfrog” hybrid Monte Carlo algorithm is a simple and effective MCMC method for fitting Bayesian generalized linear models with canonical link. The algorithm leads to large trajectories over the posterior and a rapidly mixing Markov chain, having superior performance over conventional methods in difficult problems like logistic regression with quasicomplete separation. This method offers a very attractive solution to this common problem, providing a method for identifying datasets that are quasicomplete separated, and for identifying the covariates that are at the root of the problem. The method is also quite successful in fitting generalized linear models in which the link function is extended to include a feedforward neural network. With a large number of hidden units, however, or when the dataset becomes large, the computations required in calculating the gradient in each trajectory can become very demanding. In this case, it is best to mix the algorithm with multivariate random walk Metropolis—Hastings. However, this entails very little additional programming work.     

Markov Chain Sampling Methods for Dirichlet Process Mixture Models

Abstract

This article reviews Markov chain methods for sampling from the posterior distribution of a Dirichlet process mixture model and presents two new classes of methods. One new approach is to make Metropolis—Hastings updates of the indicators specifying which mixture component is associated with each observation, perhaps supplemented with a partial form of Gibbs sampling. The other new approach extends Gibbs sampling for these indicators by using a set of auxiliary parameters. These methods are simple to implement and are more efficient than previous ways of handling general Dirichlet process mixture models with non-conjugate priors.     

On the Choice of Nugget in Kriging Modeling for Deterministic Computer Experiments

Kriging is commonly used for developing emulators as surrogates for computationally intensive simulations. One difficulty with kriging is the potential numerical instability in the computation of the inverse of the covariance matrix, which can lead to large variability and poor performance of the kriging predictor. First, we study some causes of ill-conditioning in kriging. We then study the use of nugget in kriging to overcome the numerical instability. Some asymptotic results on its interpolation bias and mean squared prediction errors are presented. Finally, we study the choice of the nugget parameter based on some algebraic lower bounds and use of a regularizing trace. A simulation study is performed to show the differences between kriging with and without nugget and to demonstrate the advantages of the former. This article has supplementary materials online.     

Efficient Bayesian Inference for Multivariate Probit Models With Sparse Inverse Correlation Matrices

We propose a Bayesian approach for inference in the multivariate probit model, taking into account the association structure between binary observations. We model the association through the correlation matrix of the latent Gaussian variables. Conditional independence is imposed by setting some off-diagonal elements of the inverse correlation matrix to zero and this sparsity structure is modeled using a decomposable graphical model. We propose an efficient Markov chain Monte Carlo algorithm relying on a parameter expansion scheme to sample from the resulting posterior distribution. This algorithm updates the correlation matrix within a simple Gibbs sampling framework and allows us to infer the correlation structure from the data, generalizing methods used for inference in decomposable Gaussian graphical models to multivariate binary observations. We demonstrate the performance of this model and of the Markov chain Monte Carlo algorithm on simulated and real datasets. This article has online supplementary materials.     

Residual Diagnostics for Covariate Effects in Spatial Point Process Models

For a spatial point process model in which the intensity depends on spatial covariates, we develop graphical diagnostics for validating the covariate effect term in the model, and for assessing whether another covariate should be added to the model. The diagnostics are point-process counterparts of the well-known partial residual plots (component-plus-residual plots) and added variable plots for generalized linear models. The new diagnostics can be derived as limits of these classical techniques under increasingly fine discretization, which leads to efficient numerical approximations. The diagnostics can also be recognized as integrals of the point process residuals, enabling us to prove asymptotic results. The diagnostics perform correctly in a simulation experiment. We demonstrate their utility in an application to geological exploration, in which a point pattern of gold deposits is modeled as a point process with intensity depending on the distance to the nearest geological fault. Online supplementary materials include technical proofs, computer code, and results of a simulation study.