Multiscale Exploratory Analysis of Regression Quantiles Using Quantile SiZer

The SiZer methodology proposed by Chaudhuri and Marron (1999) is a valuable tool for conducting exploratory data analysis. Since its inception different versions of SiZer have been proposed in the literature. Most of these SiZer variants are targeting the mean structure of the data, and are incapable of providing any information about the quantile composition of the data. To fill this need, this article proposes a quantile version of SiZer for the regression setting. By inspecting the SiZer maps produced by this new SiZer, real quantile structures hidden in a dataset can be more effectively revealed, while at the same time spurious features can be filtered out. The utility of this quantile SiZer is illustrated via applications to both real data and simulated examples. This article has supplementary material online.     

Bayesian Multiscale Smoothing for Making Inferences About Features in Scatterplots

A rather common problem of data analysis is to find interesting features, such as local minima, maxima, and trends in a scatterplot. Variance in the data can then be a problem and inferences about features must be made at some selected level of significance. The recently introduced SiZer technique uses a family of nonparametric smooths of the data to uncover features in a whole range of scales. To aid the analysis, a color map is generated that visualizes the inferences made about the significance of the features. The purpose of this article is to present Bayesian versions of SiZer methodology. Both an analytically solvable regression model and a fully Bayesian approach that uses Gibbs sampling are presented. The prior distributions of the smooths are based on a roughness penalty. Simulation based algorithms are proposed for making simultaneous inferences about the features in the data.     

Sizer for smoothing splines

Smoothing splines are an attractive method for scatterplot smoothing. The SiZer approach to statistical inference is adapted to this smoothing method, named SiZerSS. This allows quick and sure inference as to “which features in the smooth are really there” as opposed to “which are due to sampling artifacts”, when using smoothing splines for data analysis. Applications of SiZerSS to mode, linearity, quadraticity and monotonicity tests are illustrated using a real data example. Some small scale simulations are presented to demonstrate that the SiZerSS and the SiZerLL (the original local linear version of SiZer) often give similar performance in exploring data structure but they can not replace each other completely. Marron’s research was supported by the Dept. of Stat. and Appl. Prob., National Univ. of Singapore, and by the National Science Foundation Grant DMS-9971649. Zhang’s research was supported by the National Univ. of Singapore Academic Research grant R-155-000-023-112. The Editor, the Associate Editor, and the referees are appreciated for their invaluable comments and suggestions that help improve the article significantly.     

Spline Estimation of Discontinuous Regression Functions

Abstract

This article deals with regression function estimation when the regression function is smooth at all but a finite number of points. An important question is: How can one produce discontinuous output without knowledge of the location of discontinuity points? Unlike most commonly used smoothers that tend to blur discontinuity in the data, we need to find a smoother that can detect such discontinuity. In this article, linear splines are used to estimate discontinuous regression functions. A procedure of knot-merging is introduced for the estimation of regression functions near discontinuous points. The basic idea is to use multiple knots for spline estimates. We use an automatic procedure involving the least squares method, stepwise knot addition, stepwise basis deletion, knot-merging, and the Bayes information criterion to select the final model. The proposed method can produce discontinuous outputs. Numerical examples using both simulated and real data are given to illustrate the performance of the proposed method.     

Robust penalized regression spline fitting with application to additive mixed modeling

An increasingly popular method for smoothing noisy data is penalized regression spline fitting. In this paper a new procedure is proposed for fitting robust penalized regression splines. This procedure is computationally fast, straightforward to implement, and can be paired with any smoothing parameter selection method. In addition, it can also be extended to other settings, such as additive mixed modeling. Both simulated and real data examples are used to illustrate the effectiveness of the procedure.     

Geometrically designed,variable knot regression splines

A new method of Geometrically Designed least squares (LS) splines with variable knots, named GeDS, is proposed. It is based on the property that the spline regression function, viewed as a parametric curve, has a control polygon and, due to the shape preserving and convex hull properties, it closely follows the shape of this control polygon. The latter has vertices whose x-coordinates are certain knot averages and whose y-coordinates are the regression coefficients. Thus, manipulation of the position of the control polygon may be interpreted as estimation of the spline curve knots and coefficients. These geometric ideas are implemented in the two stages of the GeDS estimation method. In stage A, a linear LS spline fit to the data is constructed, and viewed as the initial position of the control polygon of a higher order (\(n>2\)) smooth spline curve. In stage B, the optimal set of knots of this higher order spline curve is found, so that its control polygon is as close to the initial polygon of stage A as possible and finally, the LS estimates of the regression coefficients of this curve are found. The GeDS method produces simultaneously linear, quadratic, cubic (and possibly higher order) spline fits with one and the same number of B-spline coefficients. Numerical examples are provided and further supplemental materials are available online.     

Composite Quantile Estimation in Partial Functional Linear Regression Model Based on Polynomial Spline

In this paper, we consider composite quantile regression for partial functional linear regression model with polynomial spline approximation. Under some mild conditions, the convergence rates of the estimators and mean squared prediction error, and asymptotic normality of parameter vector are obtained. Simulation studies demonstrate that the proposed new estimation method is robust and works much better than the least-squares based method when there are outliers in the dataset or the random error follows heavy-tailed distributions. Finally, we apply the proposed methodology to a spectroscopic data sets to illustrate its usefulness in practice.     

Penalized Nonparametric Scalar-on-Function Regression via Principal Coordinates

A number of classical approaches to nonparametric regression have recently been extended to the case of functional predictors. This article introduces a new method of this type, which extends intermediate-rank penalized smoothing to scalar-on-function regression. In the proposed method, which we call principal coordinate ridge regression, one regresses the response on leading principal coordinates defined by a relevant distance among the functional predictors, while applying a ridge penalty. Our publicly available implementation, based on generalized additive modeling software, allows for fast optimal tuning parameter selection and for extensions to multiple functional predictors, exponential family-valued responses, and mixed-effects models. In an application to signature verification data, principal coordinate ridge regression, with dynamic time warping distance used to define the principal coordinates, is shown to outperform a functional generalized linear model. Supplementary materials for this article are available online.     

Robust estimation for varying index coefficient models

Varying index coefficient models (VICMs) proposed by Ma and Song (J Am Stat Assoc, 2014. doi: 10.1080/01621459.2014.903185) are a new class of semiparametric models, which encompass most of the existing semiparametric models. So far, only the profile least squares method and local linear fitting were developed for the VICM, which are very sensitive to the outliers and will lose efficiency for the heavy tailed error distributions. In this paper, we propose an efficient and robust estimation procedure for the VICM based on modal regression which depends on a bandwidth. We establish the consistency and asymptotic normality of proposed estimators for index coefficients by utilizing profile spline modal regression method. The oracle property of estimators for the nonparametric functions is also established by utilizing a two-step spline backfitted local linear modal regression approach. In addition, we discuss the bandwidth selection for achieving better robustness and efficiency and propose a modified expectation–maximization-type algorithm for the proposed estimation procedure. Finally, simulation studies and a real data analysis are carried out to assess the finite sample performance of the proposed method.     

On the Construction of Optimal Monotone Cubic Spline Interpolations

In this paper we derive necessary optimality conditions for an interpolating spline function which minimizes the Holladay approximation of the energy functional and which stays monotone if the given interpolation data are monotone. To this end optimal control theory for state-restricted optimal control problems is applied. The necessary conditions yield a complete characterization of the optimal spline. In the case of two or three interpolation knots, which we call thelocalcase, the optimality conditions are treated analytically. They reduce to polynomial equations which can very easily be solved numerically. These results are used for the construction of a numerical algorithm for the optimal monotone spline in the general (global) case via Newton's method. Here, the local optimal spline serves as a favourable initial estimation for the additional grid points of the optimal spline. Some numerical examples are presented which are constructed by FORTRAN and MATLAB programs.     

Iterative weighted partial spline least squares estimation in semiparametric modeling of longitudinal data

In this paper we consider the estimating problem of a semiparametric regression modelling whenthe data are longitudinal.An iterative weighted partial spline least squares estimator(IWPSLSE)for the para-metric component is proposed which is more efficient than the weighted partial spline least squares estimator(WPSLSE)with weights constructed by using the within-group partial spline least squares residuals in the sense     

Spline-based sieve maximum likelihood estimation in the partly linear model under monotonicity constraints

We study a spline-based likelihood method for the partly linear model with monotonicity constraints. We use monotone B-splines to approximate the monotone nonparametric function and apply the generalized Rosen algorithm to compute the estimators jointly. We show that the spline estimator of the nonparametric component achieves the possible optimal rate of convergence under the smooth assumption and that the estimator of the regression parameter is asymptotically normal and efficient. Moreover, a spline-based semiparametric likelihood ratio test is established to make inference of the regression parameter. Also an observed profile information method to consistently estimate the standard error of the spline estimator of the regression parameter is proposed. A simulation study is conducted to evaluate the finite sample performance of the proposed method. The method is illustrated by an air pollution study.     

Robust Estimation in Partial Linear Mixed Model for Longitudinal Data

In this article, robust generalized estimating equation for the analysis of partial linear mixed model for longitudinal data is used. The authors approximate the nonparametric function by a regression spline. Under some regular conditions, the asymptotic properties of the estimators are obtained. To avoid the computation of high-dimensional integral, a robust Monte Carlo Newton-Raphson algorithm is used. Some simulations are carried out to study the performance of the proposed robust estimators. In addition, the authors also study the robustness and the efficiency of the proposed estimators by simulation. Finally, two real longitudinal data sets are analyzed.     

L 1 C 1 polynomial spline approximation algorithms for large data sets

In this article, we address the problem of approximating data points by C ¹-smooth polynomial spline curves or surfaces using L ₁-norm. The use of this norm helps to preserve the data shape and it reduces extraneous oscillations. In our approach, we introduce a new functional which enables to control directly the distance between the data points and the resulting spline solution. The computational complexity of the minimization algorithm is nonlinear. A local minimization method using sliding windows allows to compute approximation splines within a linear complexity. This strategy seems to be more robust than a global method when applied on large data sets. When the data are noisy, we iteratively apply this method to globally smooth the solution while preserving the data shape. This method is applied to image denoising.     

Change Trees and Mutagrams for the Visualization of Local Changes in Sequence Data

The analysis of local changes in sequence data is of interest for various applications such as the segmentation of DNA and other genetic sequences, or financial data sequences. Patterns of change that can be characterized as local jump change or slope change are of special interest. We propose simple graphical tools to visualize such patterns of local change. The concept of mode trees—developed for the visualization of local patterns in densities—is adapted to visualize patterns of local change in dependency on a threshold parameter by means of a change tree . The simultaneous visualization of scale effects, in analogy to SiZer, motivates another graphical device, the mutagram . We illustrate these concepts with several sets of sequence data.     

An algorithm for the computation of the exponential spline

Summary Procedures for the calculation of the exponential spline (spline under tension) are presented in this paper. The procedureexsplcoeff calculates the second derivatives of the exponential spline. Using the second derivatives the exponential spline can be evaluated in a stable and efficient manner by the procedureexspl. The limiting cases of the exponential spline, the cubic spline and the linear spline are included. A proceduregenerator is proposed, which computes appropriate tension parameters. The performance of the algorithm is discussed for several examples.Editor's Note: In this fascile, prepublication of algorithms from the Approximation series of the Handbook for Automatic Computation is continued. Algorithms are published in ALGOL 60 reference language as approved by the IFIP. Contributions in this series should be styled after the most recently published ones     

Fast Nonparametric Quantile Regression With Arbitrary Smoothing Methods

The calculation of nonparametric quantile regression curve estimates is often computationally intensive, as typically an expensive nonlinear optimization problem is involved. This article proposes a fast and easy-to-implement method for computing such estimates. The main idea is to approximate the costly nonlinear optimization by a sequence of well-studied penalized least squares-type nonparametric mean regression estimation problems. The new method can be paired with different nonparametric smoothing methods and can also be applied to higher dimensional settings. Therefore, it provides a unified framework for computing different types of nonparametric quantile regression estimates, and it also greatly broadens the scope of the applicability of quantile regression methodology. This wide applicability and the practical performance of the proposed method are illustrated with smoothing spline and wavelet curve estimators, for both uni- and bivariate settings. Results from numerical experiments suggest that estimates obtained from the proposed method are superior to many competitors. This article has supplementary material online.     

Bivariate Polynomial Natural Spline Interpolation Algorithms with Local Basis for Scattered Data

Because of its importance in both theory and applications, multivariate splines have attracted special attention in many fields. Based on the theory of spline functions in Hilbert spaces, bivariate polynomial natural splines for interpolating, smoothing or generalized interpolating of scattered data over an arbitrary domain are constructed with one-sided functions. However, this method is not well suited for large scale numerical applications. In this paper, a new locally supported basis for the bivariate polynomial natural spline space is constructed. Some properties of this basis are also discussed. Methods to order scattered data are shown and algorithms for bivariate polynomial natural spline interpolating are constructed. The interpolating coefficient matrix is sparse, and thus, the algorithms can be easily implemented in a computer.     

Uncertainty Quantification for Modern High-Dimensional Regression via Scalable Bayesian Methods

Tremendous progress has been made in the last two decades in the area of high-dimensional regression, especially in the “large p, small n” setting. Such sample starved settings inevitably lead to models which are potentially very unstable and hence quite unreliable. To this end, Bayesian shrinkage methods have generated a lot of recent interest in the modern high-dimensional regression and model selection context. Such methods span the wide spectrum of modern regression approaches and include among others, spike-and-slab priors, the Bayesian lasso, ridge regression, and global-local shrinkage priors such as the Horseshoe prior and the Dirichlet–Laplace prior. These methods naturally facilitate tractable uncertainty quantification and have thus been used extensively across diverse applications. A common unifying feature of these models is that the corresponding priors on the regression coefficients can be expressed as a scale mixture of normals. This property has been leveraged extensively to develop various three-step Gibbs samplers to explore the corresponding intractable posteriors. The convergence of such samplers however is very slow in high dimensions settings, making them disconnected to the very setting that they are intended to work in. To address this challenge, we propose a comprehensive and unifying framework to draw from the same family of posteriors via a class of tractable and scalable two-step blocked Gibbs samplers. We demonstrate that our proposed class of two-step blocked samplers exhibits vastly superior convergence behavior compared to the original three-step sampler in high-dimensional regimes on simulated data as well as data from a variety of applications including gene expression data, infrared spectroscopy data, and socio-economic/law enforcement data. We also provide a detailed theoretical underpinning to the new method by deriving explicit upper bounds for the (geometric) rate of convergence, and by proving that the proposed two-step sampler has superior spectral properties. Supplementary material for this article is available online.     

加密网格点二元局部基插值样条函数

1.简介 由于在理论以及应用两方面的重要性,多元样条引起了许多人的注意([6],[7]),紧支撑光滑分片多项式函数对于曲面的逼近是一个十分有效的工具。由于它们的局部支撑性,它们很容易求值;由于它们的光滑性,它们能被应用到要满足一定光滑条件的情况下;由于它们是紧支撑的,它们的线性包有很大的逼近灵活性,而且用它们构造逼近方法来解决的系统是