Divide and Recombine Approaches for Fitting Smoothing Spline Models with Large Datasets

Spline smoothing is a widely used nonparametric method that allows data to speak for themselves. Due to its complexity and flexibility, fitting smoothing spline models is usually computationally intensive which may become prohibitive with large datasets. To overcome memory and CPU limitations, we propose four divide and recombine (D&R) approaches for fitting cubic splines with large datasets. We consider two approaches to divide the data: random and sequential. For each approach of division, we consider two approaches to recombine. These D&R approaches are implemented in parallel without communication. Extensive simulations show that these D&R approaches are scalable and have comparable performance as the method that uses the whole data. The sequential D&R approaches are spatially adaptive which lead to better performance than the method that uses the whole data when the underlying function is spatially inhomogeneous.     

Ultra-analytic effect of Cauchy problem for a class of kinetic equations

The smoothing effect of the Cauchy problem for a class of kinetic equations is studied. We firstly consider the spatially homogeneous nonlinear Landau equation with Maxwellian molecules and inhomogeneous linear Fokker-Planck equation to show the ultra-analytic effects of the Cauchy problem. Those smoothing effect results are optimal and similar to heat equation. In the second part, we study a model of spatially inhomogeneous linear Landau equation with Maxwellian molecules, and show the analytic effect of the Cauchy problem.     

Priors for Bayesian adaptive spline smoothing

Adaptive smoothing has been proposed for curve-fitting problems where the underlying function is spatially inhomogeneous. Two Bayesian adaptive smoothing models, Bayesian adaptive smoothing splines on a lattice and Bayesian adaptive P-splines, are studied in this paper. Estimation is fully Bayesian and carried out by efficient Gibbs sampling. Choice of prior is critical in any Bayesian non-parametric regression method. We use objective priors on the first level parameters where feasible, specifically independent Jeffreys priors (right Haar priors) on the implied base linear model and error variance, and we derive sufficient conditions on higher level components to ensure that the posterior is proper. Through simulation, we demonstrate that the common practice of approximating improper priors by proper but diffuse priors may lead to invalid inference, and we show how appropriate choices of proper but only weakly informative priors yields satisfactory inference.     

A stochastic framework for recursive computation of spline functions: Part II,smoothing splines

Many of the optimal curve-fitting problems arising in approximation theory have the same structure as certain estimation problems involving random processes. We develop this structural correspondence for the problem of smoothing inaccurate data with splines and show that the smoothing spline is a sample function of a certain linear least-squares estimate. Estimation techniques are then used to derive a recursive algorithm for spline smoothing.     

An energy-minimization framework for monotonic cubic spline interpolation

This paper describes the use of cubic splines for interpolating monotonic data sets. Interpolating cubic splines are popular for fitting data because they use low-order polynomials and have C² continuity, a property that permits them to satisfy a desirable smoothness constraint. Unfortunately, that same constraint often violates another desirable property: monotonicity. It is possible for a set of monotonically increasing (or decreasing) data points to yield a curve that is not monotonic, i.e., the spline may oscillate. In such cases, it is necessary to sacrifice some smoothness in order to preserve monotonicity.The goal of this work is to determine the smoothest possible curve that passes through its control points while simultaneously satisfying the monotonicity constraint. We first describe a set of conditions that form the basis of the monotonic cubic spline interpolation algorithm presented in this paper. The conditions are simplified and consolidated to yield a fast method for determining monotonicity. This result is applied within an energy minimization framework to yield linear and nonlinear optimization-based methods. We consider various energy measures for the optimization objective functions. Comparisons among the different techniques are given, and superior monotonic C² cubic spline interpolation results are presented. Extensions to shape preserving splines and data smoothing are described.     

Multiresolution based adaptive schemes for second order hyperbolic PDEs in elastodynamic problems

An enhanced interpolation wavelet-based adaptive-grid scheme is implemented for simulating high gradient smooth solutions (as well as, discontinuous ones) in elastodynamic problems in domains with irregular boundary shapes. In the method, spatially adaptive smoothing is used to improve interpolation property of the solution in high gradient zones. In hyperbolic systems, in fact, there are no certain inherent regularities; hence, the erroneous adapted grid may be achieved because of small spurious oscillations in the solution domain. These oscillations, mainly formed in the vicinity of high gradient and discontinuity zones, make the adaptation procedure strongly unstable. To cover this drawback, enhanced smoothing splines are used to denoise directly non-physical oscillations in the irregular grid points, a kind of ill-posed problem. Controllable smoothing is achieved using non-uniform weight coefficients. As the smoothing splines are a kind of the Thikhonov regularization method, they work stably in irregular grid points. Regarding the Thikhonov regularization method, L-curve scheme could be used to investigate trade-off between accuracy and smoothness of the solutions. This relationship, in fact, could not be reliably captured by common computational methods. The proposed method, in general, is easy and conceptually straightforward; as all calculations are carried out in the physical domain. This concept is verified using a variety of 2D numerical examples.     

Semiparametric Estimation and Selection for Nonstationary Spatial Covariance Functions

We propose a method for estimating nonstationary spatial covariance functions by representing a spatial process as a linear combination of some local basis functions with uncorrelated random coefficients and some stationary processes, based on spatial data sampled in space with repeated measurements. By incorporating a large collection of local basis functions with various scales at various locations and stationary processes with various degrees of smoothness, the model is flexible enough to represent a wide variety of nonstationary spatial features. The covariance estimation and model selection are formulated as a regression problem with the sample covariances as the response and the covariances corresponding to the local basis functions and the stationary processes as the predictors. A constrained least squares approach is applied to select appropriate basis functions and stationary processes as well as estimate parameters simultaneously. In addition, a constrained generalized least squares approach is proposed to further account for the dependencies among the response variables. A simulation experiment shows that our method performs well in both covariance function estimation and spatial prediction. The methodology is applied to a U.S. precipitation dataset for illustration. Supplemental materials relating to the application are available online.     

Targeted smoothing parameter selection for estimating average causal effects

The non-parametric estimation of average causal effects in observational studies often relies on controlling for confounding covariates through smoothing regression methods such as kernel, splines or local polynomial regression. Such regression methods are tuned via smoothing parameters which regulates the amount of degrees of freedom used in the fit. In this paper we propose data-driven methods for selecting smoothing parameters when the targeted parameter is an average causal effect. For this purpose, we propose to estimate the exact expression of the mean squared error of the estimators. Asymptotic approximations indicate that the smoothing parameters minimizing this mean squared error converges to zero faster than the optimal smoothing parameter for the estimation of the regression functions. In a simulation study we show that the proposed data-driven methods for selecting the smoothing parameters yield lower empirical mean squared error than other methods available such as, e.g., cross-validation.     

Local likelihood estimation for nonstationary random fields

We develop a weighted local likelihood estimate for the parameters that govern the local spatial dependency of a locally stationary random field. The advantage of this local likelihood estimate is that it smoothly downweights the influence of faraway observations, works for irregular sampling locations, and when designed appropriately, can trade bias and variance for reducing estimation error. This paper starts with an exposition of our technique on the problem of estimating an unknown positive function when multiplied by a stationary random field. This example gives concrete evidence of the benefits of our local likelihood as compared to unweighted local likelihoods. We then discuss the difficult problem of estimating a bandwidth parameter that controls the amount of influence from distant observations. Finally we present a simulation experiment for estimating the local smoothness of a local Matérn random field when observing the field at random sampling locations in [0,1]². The local Matérn is a fully nonstationary random field, has a closed form covariance, can attain any degree of differentiability or Hölder smoothness and behaves locally like a stationary Matérn. We include an appendix that proves the positive definiteness of this covariance function.     

Bivariate Free Knot Splines

We consider the least squares approximation of gridded 2D data by tensor product splines with free knots. The smoothing functional to be minimized—a generalization of the univariate Schoenberg functional—is chosen in such a way that the solution of the bivariate problem separates into the solution of a sequence of univariate problems in case of fixed knots. The resulting optimization problem is a constrained separable least squares problem with tensor product structure. Based on some ideas developed by the authors for the univariate case, an efficient method for solving the specially structured 2D problem is proposed, analyzed and tested on hand of some examples from the literature.     

Automatic Smoothing With Wavelets for a Wide Class of Distributions

Wavelet-based denoising techniques are well suited to estimate spatially inhomogeneous signals. Waveshrink (Donoho and Johnstone) assumes independent Gaussian errors and equispaced sampling of the signal. Various articles have relaxed some of these assumptions, but a systematic generalization to distributions such as Poisson, binomial, or Bernoulli is missing. We consider a unifying l₁-penalized likelihood approach to regularize the maximum likelihood estimation by adding an l₁ penalty of the wavelet coefficients. Our approach works for all types of wavelets and for a range of noise distributions. We develop both an algorithm to solve the estimation problem and rules to select the smoothing parameter automatically. In particular, using results from Poisson processes, we give an explicit formula for the universal smoothing parameter to denoise Poisson measurements. Simulations show that the procedure is an improvement over other methods. An astronomy example is given.     

Multivariate splines and hyperplane arrangements

In this paper, we use the so-called conformality method of smoothing cofactor (abbr. CSC) and hyperplane arrangements to study truncated powers and box splines in R². By the relation between hyperplane arrangements and truncated powers, we give the expressions of the truncated powers. Moreover, by means of the CSC method, the least smoothness degrees of the truncated powers and the box splines on each direction of partition edges are studied.     

Spline Smoothing on Surfaces

This article presents a method for estimating functions on topologically and/or geometrically complex surfaces from possibly noisy observations. Our approach is an extension of spline smoothing, using a finite element method. The article has a substantial tutorial component: we start by reviewing smoothness measures for functions defined on surfaces, simplicial surfaces and differentiable structures on such surfaces, subdivison functions, and subdivision surfaces. After describing our method, we show results of an experiment comparing finite element approximations to exact smoothing splines on the sphere, and we give examples suggesting that generalized cross-validation is an effective way of determining the optimal degree of smoothing for function estimation on surfaces.     

Local Lagrange Interpolation with Bivariate Splines of Arbitrary Smoothness

We describe a method which can be used to interpolate function values at a set of scattered points in a planar domain using bivariate polynomial splines of any prescribed smoothness. The method starts with an arbitrary given triangulation of the data points, and involves refining some of the triangles with Clough-Tocher splits. The construction of the interpolating splines requires some additional function values at selected points in the domain, but no derivatives are needed at any point. Given n data points and a corresponding initial triangulation, the interpolating spline can be computed in just O(n) operations. The interpolation method is local and stable, and provides optimal order approximation of smooth functions.     

基于一个新的NCP函数的光滑牛顿法求解非线性互补问题

本文研究了非线性互补的光滑化问题.利用一个新的光滑NCP函数将非线性互补问题转化为等价的光滑方程组,并在此基础上建立了求解P0-函数非线性互补问题的一个完全光滑化牛顿法,获得了算法的全局收敛性和局部二次收敛性的结果.并给出数值实验验证了理论分析的正确性.     

Adaptive Order Selection for Spline Smoothing

Computational methods are presented for spline smoothing that make it practical to compute smoothing splines of degrees other than just the standard cubic case. Specifically, an order n algorithm is developed that has conceptual and practical advantages relative to classical methods. From a conceptual standpoint, the algorithm uses only standard programming techniques that do not require specialized knowledge about spline functions, methods for solving sparse equation systems or Kalman filtering. This allows for the practical development of methods for adaptive selection of both the level of smoothing and degree of the estimator. Simulation experiments are presented that show adaptive degree selection can improve estimator efficiency over the use of cubic smoothing splines. Run-time comparisons are also conducted between the proposed algorithm and a classical, band-limited, computing method for the cubic case.     

The Structural Characterization and Locally Supported Bases for Bivariate Super Splines

Super splines are bivariate splines defined on triangulations, where the smoothness enforced at the vertices is larger than the smoothness enforced across the edges. In this paper, the smoothness conditions and conformality conditions for super splines are presented. Three locally supported super splines on type-1 triangulation are presented. Moreover, the criteria to select local bases is also givenBy using local supported super spline function, a variation-diminishing operator is built. The approximation properties of the operator are also presented.     

A Retinex-based total variation approach for image segmentation and bias correction

Image segmentation methods usually suffer from intensity inhomogeneity problem caused by many factors such as spatial variations in illumination (or bias fields of imaging devices). In order to address this problem, this paper proposes a Retinex-based variational model for image segmentation and bias correction. According to Retinex theory, the input inhomogeneous image can be decoupled into illumination bias and reflectance parts. The main contribution of this paper is to consider piecewise constant of the reflectance, and thereby introduce the total variation term in the proposed model for correcting and segmenting the input image. This is different from the existing model in which the spatial smoothness of the illumination bias is employed only. The existence of the minimizers to the variational model is established. Furthermore, we develop an efficient algorithm to solve the model numerically by using the alternating minimization method. Our experimental results are reported to demonstrate the effectiveness of the proposed method, and its performance is competitive with that of the other testing methods.     

Numerical discretization-based kernel type estimation methods for ordinary differential equation models

We consider the problem of parameter estimation in both linear and nonlinear ordinary differential equation(ODE) models. Nonlinear ODE models are widely used in applications. But their analytic solutions are usually not available. Thus regular methods usually depend on repetitive use of numerical solutions which bring huge computational cost. We proposed a new two-stage approach which includes a smoothing method(kernel smoothing or local polynomial fitting) in the first stage, and a numerical discretization method(Eulers discretization method, the trapezoidal discretization method,or the Runge–Kutta discretization method) in the second stage. Through numerical simulations, we find the proposed method gains a proper balance between estimation accuracy and computational cost.Asymptotic properties are also presented, which show the consistency and asymptotic normality of estimators under some mild conditions. The proposed method is compared to existing methods in term of accuracy and computational cost. The simulation results show that the estimators with local linear smoothing in the first stage and trapezoidal discretization in the second stage have the lowest average relative errors. We apply the proposed method to HIV dynamics data to illustrate the practicability of the estimator.     

Sur le choix du paramètre d'ajustement dans le lissage par fonctions spline

Summary We consider the problem of approximating an unknown functionf, known with error atn equally spaced points of the real interval [a, b].To solve this problem, we use the natural polynomial smoothing splines. We show that the eigenvalues associated to these splines converge to the eigenvalues of a differential operator and we use this fact to obtain an algorithm, based on the Generalized Cross Validation method, to calculate the smoothing parameter.With this algorithm, we divide byn the time used by classical methods.