Applications of Hybrid Monte Carlo to Bayesian Generalized Linear Models: Quasicomplete Separation and Neural Networks

Abstract

The “leapfrog” hybrid Monte Carlo algorithm is a simple and effective MCMC method for fitting Bayesian generalized linear models with canonical link. The algorithm leads to large trajectories over the posterior and a rapidly mixing Markov chain, having superior performance over conventional methods in difficult problems like logistic regression with quasicomplete separation. This method offers a very attractive solution to this common problem, providing a method for identifying datasets that are quasicomplete separated, and for identifying the covariates that are at the root of the problem. The method is also quite successful in fitting generalized linear models in which the link function is extended to include a feedforward neural network. With a large number of hidden units, however, or when the dataset becomes large, the computations required in calculating the gradient in each trajectory can become very demanding. In this case, it is best to mix the algorithm with multivariate random walk Metropolis—Hastings. However, this entails very little additional programming work.     

Markov Chain Sampling Methods for Dirichlet Process Mixture Models

Abstract

This article reviews Markov chain methods for sampling from the posterior distribution of a Dirichlet process mixture model and presents two new classes of methods. One new approach is to make Metropolis—Hastings updates of the indicators specifying which mixture component is associated with each observation, perhaps supplemented with a partial form of Gibbs sampling. The other new approach extends Gibbs sampling for these indicators by using a set of auxiliary parameters. These methods are simple to implement and are more efficient than previous ways of handling general Dirichlet process mixture models with non-conjugate priors.     

On the Choice of Nugget in Kriging Modeling for Deterministic Computer Experiments

Kriging is commonly used for developing emulators as surrogates for computationally intensive simulations. One difficulty with kriging is the potential numerical instability in the computation of the inverse of the covariance matrix, which can lead to large variability and poor performance of the kriging predictor. First, we study some causes of ill-conditioning in kriging. We then study the use of nugget in kriging to overcome the numerical instability. Some asymptotic results on its interpolation bias and mean squared prediction errors are presented. Finally, we study the choice of the nugget parameter based on some algebraic lower bounds and use of a regularizing trace. A simulation study is performed to show the differences between kriging with and without nugget and to demonstrate the advantages of the former. This article has supplementary materials online.     

Residual Diagnostics for Covariate Effects in Spatial Point Process Models

For a spatial point process model in which the intensity depends on spatial covariates, we develop graphical diagnostics for validating the covariate effect term in the model, and for assessing whether another covariate should be added to the model. The diagnostics are point-process counterparts of the well-known partial residual plots (component-plus-residual plots) and added variable plots for generalized linear models. The new diagnostics can be derived as limits of these classical techniques under increasingly fine discretization, which leads to efficient numerical approximations. The diagnostics can also be recognized as integrals of the point process residuals, enabling us to prove asymptotic results. The diagnostics perform correctly in a simulation experiment. We demonstrate their utility in an application to geological exploration, in which a point pattern of gold deposits is modeled as a point process with intensity depending on the distance to the nearest geological fault. Online supplementary materials include technical proofs, computer code, and results of a simulation study.     

Iterative Plug-In Algorithms for SEMIFAR Models—Definition,Convergence, and Asymptotic Properties

This article proposes data-driven algorithms for fitting SEMIFAR models. The algorithms combine the data-driven estimation of the nonparametric trend and maximum likelihood estimation of the parameters. Convergence and asymptotic properties of the proposed algorithms are investigated. A large simulation study illustrates the practical performance of the methods.     

AMlet,RAMlet, and GAMlet: Automatic Nonlinear Fitting of Additive Models,Robust and Generalized,With Wavelets

A simple and yet powerful method is presented to estimate nonlinearly and nonparametrically the components of additive models using wavelets. The estimator enjoys the good statistical and computational properties of the Waveshrink scatterplot smoother and it can be efficiently computed using the block coordinate relaxation optimization technique. A rule for the automatic selection of the smoothing parameters, suitable for data mining of large datasets, is derived. The wavelet-based method is then extended to estimate generalized additive models. A primal-dual log-barrier interior point algorithm is proposed to solve the corresponding convex programming problem. Based on an asymptotic analysis, a rule for selecting the smoothing parameters is derived, enabling the estimator to be fully automated in practice. We illustrate the finite sample property with a Gaussian and a Poisson simulation.     

Hybrid Lévy Models: Design and Computational Aspects

ABSTRACT

A hybrid model is a model, where two markets are studied jointly such that stochastic dependence can be taken into account. Such a dependence is well known for equity and interest rate markets on which we focus here. Other pairs can be considered in a similar way. Two different versions of a hybrid approach are developed. Independent time-inhomogeneous Lévy processes are used as the drivers of the dynamics of interest rates and equity. In both versions, the dynamics of the interest rate side is described by an equation for the instantaneous forward rate. Dependence between the markets is generated by introducing the driver of the interest rate market as an additional term into the dynamics of equity in the first version. The second version starts with the equity dynamics and uses a corresponding construction for the interest rate side. Dependence can be quantified in both cases by a single parameter. Numerically efficient valuation formulas for interest rate and equity derivatives are developed. Using market quotes for liquidly traded assets we show that the hybrid approach can be successfully calibrated.     

American Options in the Heston Model with Stochastic Interest Rate and Its Generalizations

Abstract

We consider the Heston model with the stochastic interest rate of Cox–Ingersoll–Ross (CIR) type and more general models with stochastic volatility and interest rates depending on two CIR-factors; the price, volatility and interest rate may correlate. Time-derivative and infinitesimal generator of the process for factors that determine the dynamics of the interest rate and/or volatility are discretized. The result is a sequence of embedded perpetual options arising in the time discretization of a Markov-modulated Lévy model. Options in this sequence are solved using an iteration method based on the Wiener–Hopf factorization. Typical shapes of the early exercise boundary are shown, and good agreement of option prices with prices calculated with the Longstaff–Schwartz method and Medvedev–Scaillet asymptotic method is demonstrated.     

Bond graph model-based system mode identification and mode-dependent fault thresholds for hybrid systems

Hybrid system models exploit the modelling abstraction that fast state transitions take place instantaneously so that they encompass discrete events and the continuous time behaviour for the while of a system mode. If a system is in a certain mode, e.g. two rigid bodies stick together, then residuals of analytical redundancy relations (ARRs) within certain small bounds indicate that the system is healthy. An unobserved mode change, however, invalidates the current model for the dynamic behaviour. As a result, ARR residuals may exceed current thresholds indicating faults in system components that have not happened. The paper shows that ARR residuals derived from a bond graph cannot only serve as fault indicators but may also be used for bond graph model-based system mode identification. ARR residuals are numerically computed in an off-line simulation by coupling a bond graph of the faulty system to a non-faulty system bond graph through residual sinks. In real-time simulation, the faulty system model is to be replaced by measurements from the real system. As parameter values are uncertain, it is important to determine adaptive ARR thresholds that, given uncertain parameters, allow to decide whether the dynamic behaviour in a current system mode is the one of the healthy system so that false alarms or overlooking of true faults can be avoided. The paper shows how incremental bond graphs can be used to determine adaptive mode-dependent ARR thresholds for switched linear time-invariant systems with uncertain parameters in order to support robust fault detection. Bond graph-based hybrid system mode identification as well as the determination of adaptive fault thresholds is illustrated by application to a power electronic system easy to survey. Some simulation results have been analytically validated.     

Case studies in dynamic modelling of large-scale wastewater treatment plants

Several large-scale wastewater treatment plants have been modelled successfully using an extensive modelling library in the General Purpose Simulator, GPS-X. The mathematical models included all important processes on the selected wastewater treatment plants, i.e. primary sedimentation, activated sludge biokinetics and final clarification.The models have been used in different areas, such as: a)analysis of operational scenarios; b) determenation of sustained and peak capacity of plants(rated plant capacity) and c) investigation of plant expansion scenarios. Out of more than twenty documented cases four, involving simulation studies on large-scale wastewater plants are discussed.