Priors for Bayesian adaptive spline smoothing

Adaptive smoothing has been proposed for curve-fitting problems where the underlying function is spatially inhomogeneous. Two Bayesian adaptive smoothing models, Bayesian adaptive smoothing splines on a lattice and Bayesian adaptive P-splines, are studied in this paper. Estimation is fully Bayesian and carried out by efficient Gibbs sampling. Choice of prior is critical in any Bayesian non-parametric regression method. We use objective priors on the first level parameters where feasible, specifically independent Jeffreys priors (right Haar priors) on the implied base linear model and error variance, and we derive sufficient conditions on higher level components to ensure that the posterior is proper. Through simulation, we demonstrate that the common practice of approximating improper priors by proper but diffuse priors may lead to invalid inference, and we show how appropriate choices of proper but only weakly informative priors yields satisfactory inference.     

Robust penalized regression spline fitting with application to additive mixed modeling

An increasingly popular method for smoothing noisy data is penalized regression spline fitting. In this paper a new procedure is proposed for fitting robust penalized regression splines. This procedure is computationally fast, straightforward to implement, and can be paired with any smoothing parameter selection method. In addition, it can also be extended to other settings, such as additive mixed modeling. Both simulated and real data examples are used to illustrate the effectiveness of the procedure.     

Selecting the Number of Knots for Penalized Splines

Penalized splines, or P-splines, are regression splines fit by least-squares with a roughness penalty.P-splines have much in common with smoothing splines, but the type of penalty used with a P-spline is somewhat more general than for a smoothing spline. Also, the number and location of the knots of a P-spline is not fixed as with a smoothing spline. Generally, the knots of a P-spline are at fixed quantiles of the independent variable and the only tuning parameters to choose are the number of knots and the penalty parameter. In this article, the effects of the number of knots on the performance of P-splines are studied. Two algorithms are proposed for the automatic selection of the number of knots. The myopic algorithm stops when no improvement in the generalized cross-validation statistic (GCV) is noticed with the last increase in the number of knots. The full search examines all candidates in a fixed sequence of possible numbers of knots and chooses the candidate that minimizes GCV.The myopic algorithm works well in many cases but can stop prematurely. The full-search algorithm worked well in all examples examined. A Demmler–Reinsch type diagonalization for computing univariate and additive P-splines is described. The Demmler–Reinsch basis is not effective for smoothing splines because smoothing splines have too many knots. For P-splines, however, the Demmler–Reinsch basis is very useful for super-fast generalized cross-validation.     

Spatially Adaptive Bayesian Penalized Regression Splines (P-splines)

In this article we study penalized regression splines (P-splines), which are low-order basis splines with a penalty to avoid undersmoothing. Such P-splines are typically not spatially adaptive, and hence can have trouble when functions are varying rapidly. Our approach is to model the penalty parameter inherent in the P-spline method as a heteroscedastic regression function. We develop a full Bayesian hierarchical structure to do this and use Markov chain Monte Carlo techniques for drawing random samples from the posterior for inference. The advantage of using a Bayesian approach to P-splines is that it allows for simultaneous estimation of the smooth functions and the underlying penalty curve in addition to providing uncertainty intervals of the estimated curve. The Bayesian credible intervals obtained for the estimated curve are shown to have pointwise coverage probabilities close to nominal. The method is extended to additive models with simultaneous spline-based penalty functions for the unknown functions. In simulations, the approach achieves very competitive performance with the current best frequentist P-spline method in terms of frequentist mean squared error and coverage probabilities of the credible intervals, and performs better than some of the other Bayesian methods.     

Statistical application of barycentric rational interpolants: an alternative to splines

Spline curves, originally developed by numerical analysts for interpolation, are widely used in statistical work, mainly as regression splines and smoothing splines. Barycentric rational interpolants have recently been developed by numerical analysts, but have yet seen very few statistical applications. We give the necesssary information to enable the reader to use barycentric rational interpolants, including a suggestion for a Bayesian prior distribution, and explore the possible statistical use of barycentric interpolants as an alternative to splines. We give the all the necessary formulae, compare the numerical accuracy to splines for some Monte-Carlo datasets, and apply both regression splines and barycentric interpolants to two real datasets. We also discuss the application of these interpolants to data smoothing, where smoothing splines would normally be used, and exemplify the use of smoothing interpolants with another real dataset. Our conclusion is that barycentric interpolants are as accurate as splines, and no more difficult to understand and program. They offer a viable alternative methodology.     

Generalized Nonparametric Mixed Effects Models

Generalized linear mixed effects models (GLMM) provide useful tools for correlated and/or over-dispersed non-Gaussian data. This article considers generalized nonparametric mixed effects models (GNMM), which relax the rigid linear assumption on the conditional predictor in a GLMM. We use smoothing splines to model fixed effects. The random effects are general and may also contain stochastic processes corresponding to smoothing splines. We show how to construct smoothing spline ANOVA (SS ANOVA) decompositions for the predictor function. Components in a SS ANOVA decomposition have nice interpretations as main effects and interactions. Experimental design considerations help determine which components are fixed or random. We estimate all parameters and spline functions using stochastic approximation with Markov chain Monte Carlo (MCMC). As iteration increases we increase the MCMC sample size and decrease the step-size of the parameter update. This approach guarantees convergence of the estimates to the expected fixed points. We evaluate our methods through a simulation study.     

Objective Bayesian Model Selection in Generalized Additive Models With Penalized Splines

We propose an objective Bayesian approach to the selection of covariates and their penalized splines transformations in generalized additive models. The methodology is based on a combination of continuous mixtures of g-priors for model parameters and a multiplicity-correction prior for the models themselves. We introduce our approach in the normal model and extend it to nonnormal exponential families. A simulation study and an application with binary outcome is provided. An efficient implementation is available in the R package hypergsplines. Supplementary materials for this article are available online.     

Bayesian Semiparametric Density Deconvolution in the Presence of Conditionally Heteroscedastic Measurement Errors

We consider the problem of estimating the density of a random variable when precise measurements on the variable are not available, but replicated proxies contaminated with measurement error are available for sufficiently many subjects. Under the assumption of additive measurement errors this reduces to a problem of deconvolution of densities. Deconvolution methods often make restrictive and unrealistic assumptions about the density of interest and the distribution of measurement errors, for example, normality and homoscedasticity and thus independence from the variable of interest. This article relaxes these assumptions and introduces novel Bayesian semiparametric methodology based on Dirichlet process mixture models for robust deconvolution of densities in the presence of conditionally heteroscedastic measurement errors. In particular, the models can adapt to asymmetry, heavy tails, and multimodality. In simulation experiments, we show that our methods vastly outperform a recent Bayesian approach based on estimating the densities via mixtures of splines. We apply our methods to data from nutritional epidemiology. Even in the special case when the measurement errors are homoscedastic, our methodology is novel and dominates other methods that have been proposed previously. Additional simulation results, instructions on getting access to the dataset and R programs implementing our methods are included as part of online supplementary materials.     

Diagnostics for penalized least-squares estimators

Diagnostic methods for a class of penalized least-squares estimators are derived from a Bayesian perspective. The class of estimators considered includes generalized ridge estimators, partial splines and thin plate smoothing splines. The proposed diagnostics include scaled residuals, leverage values and various measures of influence.     

用广义交互确认方法选择良好参数进行多元多项式散乱数据自然样条光顺

1．简介多元样条函数在多元逼近中发挥很大作用，已有数量相当多的综合报告和研究论文正式发表，就在1996年6月在法国召开的第三届国际曲线与曲面会议上便有不少多元样条方面的报告，不过总的感觉是仍然缺乏对噪声数据特别是散乱数据的有效光顺方法．李岳生、崔锦泰、关履泰、胡日章等讨论广义调配样条与张量积函数，并用希氏空间样条方法处理多元散乱数据样条插值与光顺，提出多元多项式自然样条，推广了相应一元的结果．我们知道，在样条光顺中有一个如何选择参数的问题，用广义交互确认方法（generalizedcross－validation，以下简称GC…     

Divide and Recombine Approaches for Fitting Smoothing Spline Models with Large Datasets

Spline smoothing is a widely used nonparametric method that allows data to speak for themselves. Due to its complexity and flexibility, fitting smoothing spline models is usually computationally intensive which may become prohibitive with large datasets. To overcome memory and CPU limitations, we propose four divide and recombine (D&R) approaches for fitting cubic splines with large datasets. We consider two approaches to divide the data: random and sequential. For each approach of division, we consider two approaches to recombine. These D&R approaches are implemented in parallel without communication. Extensive simulations show that these D&R approaches are scalable and have comparable performance as the method that uses the whole data. The sequential D&R approaches are spatially adaptive which lead to better performance than the method that uses the whole data when the underlying function is spatially inhomogeneous.     

On spline functions and statistical regularization of ill-posed problems

Interpolating natural splines are used for the algebraization and smoothing regularization of linear Fredholm integral equations of the first kind. A simplified version of statistical regularization is presented and, in turn, applied to data graduation by smoothing natural splines.     

Sizer for smoothing splines

Smoothing splines are an attractive method for scatterplot smoothing. The SiZer approach to statistical inference is adapted to this smoothing method, named SiZerSS. This allows quick and sure inference as to “which features in the smooth are really there” as opposed to “which are due to sampling artifacts”, when using smoothing splines for data analysis. Applications of SiZerSS to mode, linearity, quadraticity and monotonicity tests are illustrated using a real data example. Some small scale simulations are presented to demonstrate that the SiZerSS and the SiZerLL (the original local linear version of SiZer) often give similar performance in exploring data structure but they can not replace each other completely. Marron’s research was supported by the Dept. of Stat. and Appl. Prob., National Univ. of Singapore, and by the National Science Foundation Grant DMS-9971649. Zhang’s research was supported by the National Univ. of Singapore Academic Research grant R-155-000-023-112. The Editor, the Associate Editor, and the referees are appreciated for their invaluable comments and suggestions that help improve the article significantly.     

Analysis of Multi-Unit Variance Components Models with State Space Profiles

We apply the Kalman Filter to the analysis of multi-unit variance components models where each unit's response profile follows a state space model. We use mixed model results to obtain estimates of unit-specific random effects, state disturbance terms and residual noise terms. We use the signal extraction approach to smooth individual profiles. We show how to utilize the Kalman Filter to efficiently compute the restricted loglikelihood of the model. For the important special case where each unit's response profile follows a continuous structural time series model with known transition matrix we derive an EM algorithm for the restricted maximum likelihood (REML) estimation of the variance components. We present details for the case where individual profiles are modeled as local polynomial trends or polynomial smoothing splines.     

函数系数线性自回归模型的样条估计

基于多项式样条全局光滑方法,建立函数系数线性自回归模型中系数函数的样条估计.在适当条件下,证明了系数函数多项式样条估计的相合性,并给出了它们的收敛速度.模拟例子验证了理论结果的正确性.     

Fast and Stable Multiple Smoothing Parameter Selection in Smoothing Spline Analysis of Variance Models With Large Samples

The current parameterization and algorithm used to fit a smoothing spline analysis of variance (SSANOVA) model are computationally expensive, making a generalized additive model (GAM) the preferred method for multivariate smoothing. In this article, we propose an efficient reparameterization of the smoothing parameters in SSANOVA models, and a scalable algorithm for estimating multiple smoothing parameters in SSANOVAs. To validate our approach, we present two simulation studies comparing our reparameterization and algorithm to implementations of SSANOVAs and GAMs that are currently available in R. Our simulation results demonstrate that (a) our scalable SSANOVA algorithm outperforms the currently used SSANOVA algorithm, and (b) SSANOVAs can be a fast and reliable alternative to GAMs. We also provide an example with oceanographic data that demonstrates the practical advantage of our SSANOVA framework. Supplementary materials that are available online can be used to replicate the analyses in this article.     

非线性再生散度随机效应模型的贝叶斯分析

非线性再生散度随机效应模型是一类非常广泛的统计模型，包括了线性随机效应模型、非线性随机效应模型、广义线性随机效应模型和指数族非线性随机效应模型等．本文研究非线性再生散度随机效应模型的贝叶斯分析．通过视随机效应为缺失数据以及应用结合Gibbs抽样技术和Metropolis-Hastings算法（简称MH算法）的混合算法获得了模型参数与随机效应的同时贝叶斯估计．最后，用一个模拟研究和一个实际例子说明上述算法的可行眭．     

Semiparametric estimation of mean and variance functions for non-Gaussian data

Flexible modelling of the response variance in regression is interesting for understanding the causes of variability in the responses, and is crucial for efficient estimation and correct inference for mean parameters. In this paper we describe methods for mean and variance estimation where the responses are modelled using the double exponential family of distributions and mean and dispersion parameters are described as an additive function of predictors. The additive terms in the model are represented by penalized splines. A simple and unified computational methodology is presented for carrying out the calculations required for Bayesian inference in this class of models based on an adaptive Metropolis algorithm. Application of the adaptive Metropolis algorithm is fully automatic and does not require any kind of pretuning runs. The methodology presented provides flexible methods for modelling heterogeneous Gaussian data, as well as overdispersed and underdispersed count data. Performance is considered in a variety of examples involving real and simulated data sets.     

广义变系数模型的Bayesian B样条估计

提出了广义变系数模型函数系数的一种新的估计方法．我们用B样条函数逼近函数系数,不具体选择节点的个数,而是节点个数取均匀的无信息先验,样条函数系数取正态先验,用Bayesian模型平均的方法估计各个函数系数．这种估计方法一个主要特点是允许各个函数系数所需节点个数的后验分布不同,因此允许不同函数系数使用不同的光滑参数．另外,本文还给出了Bayesian B样条估计的计算方法,并通过模拟例子,说明广义变系数模型的函数系数可以由Bayesian B样条估计方法得到很好的估计．     

Local asymptotic behavior of regression splines for marginal semiparametric models with longitudinal data

In this paper, we study the local asymptotic behavior of the regression spline estimator in the framework of marginal semiparametric model. Similarly to Zhu, Fung and He (2008), we give explicit expression for the asymptotic bias of regression spline estimator for nonparametric function f. Our results also show that the asymptotic bias of the regression spline estimator does not depend on the working covariance matrix, which distinguishes the regression splines from the smoothing splines and the seemingly u...