Variable Selection in Varying-Coefficient Models Using P-Splines

In this article, we consider nonparametric smoothing and variable selection in varying-coefficient models. Varying-coefficient models are commonly used for analyzing the time-dependent effects of covariates on responses measured repeatedly (such as longitudinal data). We present the P-spline estimator in this context and show its estimation consistency for a diverging number of knots (or B-spline basis functions). The combination of P-splines with nonnegative garrote (which is a variable selection method) leads to good estimation and variable selection. Moreover, we consider APSO (additive P-spline selection operator), which combines a P-spline penalty with a regularization penalty, and show its estimation and variable selection consistency. The methods are illustrated with a simulation study and real-data examples. The proofs of the theoretical results as well as one of the real-data examples are provided in the online supplementary materials.     

Spatially Adaptive Bayesian Penalized Regression Splines (P-splines)

In this article we study penalized regression splines (P-splines), which are low-order basis splines with a penalty to avoid undersmoothing. Such P-splines are typically not spatially adaptive, and hence can have trouble when functions are varying rapidly. Our approach is to model the penalty parameter inherent in the P-spline method as a heteroscedastic regression function. We develop a full Bayesian hierarchical structure to do this and use Markov chain Monte Carlo techniques for drawing random samples from the posterior for inference. The advantage of using a Bayesian approach to P-splines is that it allows for simultaneous estimation of the smooth functions and the underlying penalty curve in addition to providing uncertainty intervals of the estimated curve. The Bayesian credible intervals obtained for the estimated curve are shown to have pointwise coverage probabilities close to nominal. The method is extended to additive models with simultaneous spline-based penalty functions for the unknown functions. In simulations, the approach achieves very competitive performance with the current best frequentist P-spline method in terms of frequentist mean squared error and coverage probabilities of the credible intervals, and performs better than some of the other Bayesian methods.     

Characterization Theorem for Best Polynomial Spline Approximation with Free Knots,Variable Degree and Fixed Tails

In this paper, we derive a necessary condition for a best approximation by piecewise polynomial functions of varying degree from one interval to another. Based on these results, we obtain a characterization theorem for the polynomial splines with fixed tails, that is the value of the spline is fixed in one or more knots (external or internal). We apply nonsmooth nonconvex analysis to obtain this result, which is also a necessary and sufficient condition for inf-stationarity in the sense of Demyanov–Rubinov. This paper is an extension of a paper where similar conditions were obtained for free tails splines. The main results of this paper are essential for the development of a Remez-type algorithm for free knot spline approximation.     

Penalized Estimation of Free-Knot Splines

Abstract

Polynomial splines are often used in statistical regression models for smooth response functions. When the number and location of the knots are optimized, the approximating power of the spline is improved and the model is nonparametric with locally determined smoothness. However, finding the optimal knot locations is an historically difficult problem. We present a new estimation approach that improves computational properties by penalizing coalescing knots. The resulting estimator is easier to compute than the unpenalized estimates of knot positions, eliminates unnecessary “corners” in the fitted curve, and in simulation studies, shows no increase in the loss. A number of GCV and AIC type criteria for choosing the number of knots are evaluated via simulation.     

Smoothing noisy data for irregular regions using penalized bivariate splines on triangulations

The penalized spline method has been widely used for estimating univariate smooth functions based on noisy data. This paper studies its extension to the two-dimensional case. To accommodate the need of handling data distributed on irregular regions, we consider bivariate splines defined on triangulations. Penalty functions based on the second-order derivatives are employed to regularize the spline fit and generalized cross-validation is used to select the penalty parameters. A simulation study shows that the penalized bivariate spline method is competitive to some well-established two-dimensional smoothers. The method is also illustrated using a real dataset on Texas temperature.     

对部分线性模型用惩罚最小二乘估计时最优光滑化的注记

部分线性模型也就是响应变量关于一个或者多个协变量是线性的, 但对于其他的协变量是非线性的关系\bd 对于部分线性模型中的参数和非参数部分的估计方法, 惩罚最小二乘估计是重要的估计方法之一\bd 对于这种估计方法, 广义交叉验证法提供了一种确定光滑参数的方法\bd 但是, 在部分线性模型中, 用广义交叉验证法确定光滑参数的最优性还没有被证明\bd 本文证明了利用惩罚最小二乘估计对于部分线性模型估计时, 用广义交叉验证法选择光滑参数的最优性\bd 通过模拟验证了本文中所提出的用广义交叉验证法选择光滑参数具有很好的效果, 同时, 本文在模拟部分比较了广义交叉验证和最小二乘交叉验证的优劣.     

Generalized Nonparametric Mixed Effects Models

Generalized linear mixed effects models (GLMM) provide useful tools for correlated and/or over-dispersed non-Gaussian data. This article considers generalized nonparametric mixed effects models (GNMM), which relax the rigid linear assumption on the conditional predictor in a GLMM. We use smoothing splines to model fixed effects. The random effects are general and may also contain stochastic processes corresponding to smoothing splines. We show how to construct smoothing spline ANOVA (SS ANOVA) decompositions for the predictor function. Components in a SS ANOVA decomposition have nice interpretations as main effects and interactions. Experimental design considerations help determine which components are fixed or random. We estimate all parameters and spline functions using stochastic approximation with Markov chain Monte Carlo (MCMC). As iteration increases we increase the MCMC sample size and decrease the step-size of the parameter update. This approach guarantees convergence of the estimates to the expected fixed points. We evaluate our methods through a simulation study.     

Spline Smoothing on Surfaces

This article presents a method for estimating functions on topologically and/or geometrically complex surfaces from possibly noisy observations. Our approach is an extension of spline smoothing, using a finite element method. The article has a substantial tutorial component: we start by reviewing smoothness measures for functions defined on surfaces, simplicial surfaces and differentiable structures on such surfaces, subdivison functions, and subdivision surfaces. After describing our method, we show results of an experiment comparing finite element approximations to exact smoothing splines on the sphere, and we give examples suggesting that generalized cross-validation is an effective way of determining the optimal degree of smoothing for function estimation on surfaces.     

Evolutionary computation for optimal knots allocation in smoothing splines

In this paper, a novel methodology is presented for optimal placement and selections of knots, for approximating or fitting curves to data, using smoothing splines. It is well-known that the placement of the knots in smoothing spline approximation has an important and considerable effect on the behavior of the final approximation [1]. However, as pointed out in [2], although spline for approximation is well understood, the knot placement problem has not been dealt with adequately. In the specialized bibliography, several methodologies have been presented for selection and optimization of parameters within B-spline, using techniques based on selecting knots called dominant points, adaptive knots placement, by data selection process, optimal control over the knots, and recently, by using paradigms from computational intelligent, and Bayesian model for automatically determining knot placement in spline modeling. However, a common two-step knot selection strategy, frequently used in the bibliography, is an homogeneous distribution of the knots or equally spaced approach [3].     

Uniform Approximation by the Highest Defect Continuous Polynomial Splines: Necessary and Sufficient Optimality Conditions and Their Generalisations

In this paper necessary and sufficient optimality conditions for uniform approximation of continuous functions by polynomial splines with fixed knots are derived. The obtained results are generalisations of the existing results obtained for polynomial approximation and polynomial spline approximation. The main result is two-fold. First, the generalisation of the existing results to the case when the degree of the polynomials, which compose polynomial splines, can vary from one subinterval to another. Second, the construction of necessary and sufficient optimality conditions for polynomial spline approximation with fixed values of the splines at one or both borders of the corresponding approximation interval.     

Local modification of Pythagorean-hodograph quintic spline curves using the B-spline form

The problems of determining the B–spline form of a C ² Pythagorean–hodograph (PH) quintic spline curve interpolating given points, and of using this form to make local modifications, are addressed. To achieve the correct order of continuity, a quintic B–spline basis constructed on a knot sequence in which each (interior) knot is of multiplicity 3 is required. C ² quintic bases on uniform triple knots are constructed for both open and closed C ² curves, and are used to derive simple explicit formulae for the B–spline control points of C ² PH quintic spline curves. These B-spline control points are verified, and generalized to the case of non–uniform knots, by applying a knot removal scheme to the Bézier control points of the individual PH quintic spline segments, associated with a set of six–fold knots. Based on the B–spline form, a scheme for the local modification of planar PH quintic splines, in response to a control point displacement, is proposed. Only two contiguous spline segments are modified, but to preserve the PH nature of the modified segments, the continuity between modified and unmodified segments must be relaxed from C ² to C ¹. A number of computed examples are presented, to compare the shape quality of PH quintic and “ordinary” cubic splines subject to control point modifications.     

Practical use of robust GCV and modified GCV for spline smoothing

Generalized cross-validation (GCV) is a popular parameter selection criterion for spline smoothing of noisy data, but it sometimes yields a severely undersmoothed estimate, especially if the sample size is small. Robust GCV (RGCV) and modified GCV are stable extensions of GCV, with the degree of stabilization depending on a parameter \(\gamma \in (0,1)\) for RGCV and on a parameter \(\rho >1\) for modified GCV. While there are favorable asymptotic results about the performance of RGCV and modified GCV, little is known for finite samples. In a large simulation study with cubic splines, we investigate the behavior of the optimal values of \(\gamma \) and \(\rho \), and identify simple practical rules to choose them that are close to optimal. With these rules, both RGCV and modified GCV perform significantly better than GCV. The performance is defined in terms of the Sobolev error, which is shown by example to be more consistent with a visual assessment of the fit than the prediction error (average squared error). The results are consistent with known asymptotic results.     

Adaptive Free-Knot Splines

This article proposes a function estimation procedure using free-knot splines as well as an associated algorithm for implementation in nonparametric regression. In contrast to conventional splines with knots confined to distinct design points, the splines allow selection of knot numbers and replacement of knots at any location and repeated knots at the same location. This exibility leads to an adaptive spline estimator that adapts any function with inhomogeneous smoothness, including discontinuity, which substantially improves the representation power of splines. Due to uses of a large class of spline functions, knot selection becomes extremely important. The existing knot selection schemes—such as stepwise selection—suffer the difficulty of knot confounding and are unsuitable for our purpose. A new knot selection scheme is proposed using an evolutionary Monte Carlo algorithm and an adaptive model selection criterion. The evolutionary algorithm locates the optimal knots accurately, whereas the adaptive model selection strategy guards against the selection error in searching through a large candidate knot space. The performance of the procedure is examined and illustrated via simulations. The procedure provides a significant improvement in performance over the other competing adaptive methods proposed in the literature. Finally, usefulness of the procedure is illustrated by an application to actual dataset.     

用广义交互确认方法选择良好参数进行多元多项式散乱数据自然样条光顺

1．简介多元样条函数在多元逼近中发挥很大作用，已有数量相当多的综合报告和研究论文正式发表，就在1996年6月在法国召开的第三届国际曲线与曲面会议上便有不少多元样条方面的报告，不过总的感觉是仍然缺乏对噪声数据特别是散乱数据的有效光顺方法．李岳生、崔锦泰、关履泰、胡日章等讨论广义调配样条与张量积函数，并用希氏空间样条方法处理多元散乱数据样条插值与光顺，提出多元多项式自然样条，推广了相应一元的结果．我们知道，在样条光顺中有一个如何选择参数的问题，用广义交互确认方法（generalizedcross－validation，以下简称GC…     

Adaptive Order Selection for Spline Smoothing

Computational methods are presented for spline smoothing that make it practical to compute smoothing splines of degrees other than just the standard cubic case. Specifically, an order n algorithm is developed that has conceptual and practical advantages relative to classical methods. From a conceptual standpoint, the algorithm uses only standard programming techniques that do not require specialized knowledge about spline functions, methods for solving sparse equation systems or Kalman filtering. This allows for the practical development of methods for adaptive selection of both the level of smoothing and degree of the estimator. Simulation experiments are presented that show adaptive degree selection can improve estimator efficiency over the use of cubic smoothing splines. Run-time comparisons are also conducted between the proposed algorithm and a classical, band-limited, computing method for the cubic case.     

A Note on the Optimality of Generalized Cross-validation Bandwidth Selection in Partially Linear Models with Kernel Smoothing Estimator

The issue of selection of bandwidth in kernel smoothing method is considered within the context of partially linear models, hi this paper, we study the asymptotic behavior of the bandwidth choice based on generalized cross-validation （CCV） approach and prove that this bandwidth choice is asymptotically optimal. Numerical simulation are also conducted to investigate the empirical performance of generalized cross-valldation.     

B-splines with homogenized knots

The use of homogenized knots for manipulating univariate polynomials by blossoming algorithms is extended to piecewise polynomials. A generalization of the B-spline to homogenized knots is studied. The new B-spline retains the triangular blossoming algorithms for evaluation, differentiation and knot insertion. Moreover, the B-spline is locally supported and a Marsden’s identity exists. Spaces of natural splines and certain polynomial spline spaces with more general continuity properties than ordinary splines have bases of B-splines over homogenized knots. Applications to nonpolynomial splines such as trigonometric and hyperbolic splines are made.     

Smoothing splines using compactly supported, positive definite, radial basis functions

In this paper, we develop a fast algorithm for a smoothing spline estimator in multivariate regression. To accomplish this, we employ general concepts associated with roughness penalty methods in conjunction with the theory of radial basis functions and reproducing kernel Hilbert spaces. It is shown that through the use of compactly supported radial basis functions it becomes possible to recover the band structured matrix feature of univariate spline smoothing and thereby obtain a fast computational algorithm. Given n data points in R ², the new algorithm has complexity O(n ²) compared to O(n ³), the order for the thin plate multivariate smoothing splines.     

A stochastic framework for recursive computation of spline functions: Part II,smoothing splines

Many of the optimal curve-fitting problems arising in approximation theory have the same structure as certain estimation problems involving random processes. We develop this structural correspondence for the problem of smoothing inaccurate data with splines and show that the smoothing spline is a sample function of a certain linear least-squares estimate. Estimation techniques are then used to derive a recursive algorithm for spline smoothing.     

B-splines with homogenized knots

The use of homogenized knots for manipulating univariate polynomials by blossoming algorithms is extended to piecewise polynomials. A generalization of the B-spline to homogenized knots is studied. The new B-spline retains the triangular blossoming algorithms for evaluation, differentiation and knot insertion. Moreover, the B-spline is locally supported and a Marsden’s identity exists. Spaces of natural splines and certain polynomial spline spaces with more general continuity properties than ordinary splines have bases of B-splines over homogenized knots. Applications to nonpolynomial splines such as trigonometric and hyperbolic splines are made.