HERMITE SCATTERED DATA FITTING BY THE PENALIZED LEAST SQUARES METHOD

Given a set of scattered data with derivative values. If the data is noisy or there is an extremely large number of data, we use an extension of the penalized least squares method of von Golitschek and Schumaker [Serdica, 18 （2002）, pp.1001-1020] to fit the data. We show that the extension of the penalized least squares method produces a unique spline to fit the data. Also we give the error bound for the extension method. Some numerical examples are presented to demonstrate the effectiveness of the proposed method.     

Comparisons between simultaneous and componentwise splines for varying coefficient models

In this paper, we study the properties of the simultaneous and componentwise splines for the varying coefficient model with repeatedly measured (longitudinal) dependent variable and time invariant covariates. The proposed simultaneous smoothing spline estimators are mainly obtained from the penalized least squares with adjustment for the variations of covariates in the penalized terms. We do this mainly to avoid the penalized terms being influenced by the scales of the covariates and the random smoothing parameters appearing in the estimators, which complicates the derivation of the asymptotic properties of the estimators. It is shown in this study that our estimators have smaller variances than the componentwise ones. Through a Monte Carlo simulation and two empirical examples, the simultaneous smoothing splines are all found to be more accurate in the variances.     

Bridge estimators and the adaptive lasso under heteroscedasticity

In this paper we investigate penalized least squares methods in linear regression models with heteroscedastic error structure. It is demonstrated that the basic properties with respect to model selection and parameter estimation of bridge estimators, Lasso and adaptive Lasso do not change if the assumption of homoscedasticity is violated. However, these estimators do not have oracle properties in the sense of Fan and Li (2001) if the oracle is based on weighted least squares. In order to address this problem we introduce weighted penalized least squares methods and demonstrate their advantages by asymptotic theory and by means of a simulation study.     

Backward perturbation analysis for scaled total least‐squares problems

The scaled total least‐squares (STLS) method unifies the ordinary least‐squares (OLS), the total least‐squares (TLS), and the data least‐squares (DLS) methods. In this paper we perform a backward perturbation analysis of the STLS problem. This also unifies the backward perturbation analyses of the OLS, TLS and DLS problems. We derive an expression for an extended minimal backward error of the STLS problem. This is an asymptotically tight lower bound on the true minimal backward error. If the given approximate solution is close enough to the true STLS solution (as is the goal in practice), then the extended minimal backward error is in fact the minimal backward error. Since the extended minimal backward error is expensive to compute directly, we present a lower bound on it as well as an asymptotic estimate for it, both of which can be computed or estimated more efficiently. Our numerical examples suggest that the lower bound gives good order of magnitude approximations, while the asymptotic estimate is an excellent estimate. We show how to use our results to easily obtain the corresponding results for the OLS and DLS problems in the literature. Copyright © 2009 John Wiley & Sons, Ltd.     

半参数回归模型的稳健补偿最小二乘估计

本文研究了一类半参数回归模型，利用稳健补偿最小二乘估计法，得到了稳健补偿最小二乘估计量，以及它们的影响函数及渐近方差一协方差，对结果的分析表明了该法优于补偿最小二乘法，而且具有稳定性．     

On the impact of model selection on predictor identification and parameter inference

We assessed the ability of several penalized regression methods for linear and logistic models to identify outcome-associated predictors and the impact of predictor selection on parameter inference for practical sample sizes. We studied effect estimates obtained directly from penalized methods (Algorithm 1), or by refitting selected predictors with standard regression (Algorithm 2). For linear models, penalized linear regression, elastic net, smoothly clipped absolute deviation (SCAD), least angle regression and LASSO had a low false negative (FN) predictor selection rates but false positive (FP) rates above 20 % for all sample and effect sizes. Partial least squares regression had few FPs but many FNs. Only relaxo had low FP and FN rates. For logistic models, LASSO and penalized logistic regression had many FPs and few FNs for all sample and effect sizes. SCAD and adaptive logistic regression had low or moderate FP rates but many FNs. 95 % confidence interval coverage of predictors with null effects was approximately 100 % for Algorithm 1 for all methods, and 95 % for Algorithm 2 for large sample and effect sizes. Coverage was low only for penalized partial least squares (linear regression). For outcome-associated predictors, coverage was close to 95 % for Algorithm 2 for large sample and effect sizes for all methods except penalized partial least squares and penalized logistic regression. Coverage was sub-nominal for Algorithm 1. In conclusion, many methods performed comparably, and while Algorithm 2 is preferred to Algorithm 1 for estimation, it yields valid inference only for large effect and sample sizes.     

Mean quadratic error of an estimate of splines of first order in a model of linear regression

We study the mean quadratic error of an estimate of splines of the first order, which is obtained by the method of least squares under the assumption that the data represents a superposition of proper values of a spline and a white noise. A quantitative formula for the quadratic mean error is found and its asymptotics is investigated.Translated from Ukrainskii Matematicheskii Zhurnal, Vol. 49, No. 3, pp. 429–432, March, 1991.     

Mean quadratic error of an estimate of splines of first order in a model of linear regression

We study the mean quadratic error of an estimate of splines of the first order, which is obtained by the method of least squares under the assumption that the data represents a superposition of proper values of a spline and a white noise. A quantitative formula for the quadratic mean error is found and its asymptotics is investigated.Translated from Ukrainskii Matematicheskii Zhurnal, Vol. 49, No. 3, pp. 429–432, March, 1991.     

Market share analysis using semi-parametric attraction models

Attraction models used to analyze the effects of marketing instruments on market share hitherto assume certain strict functional forms. We introduce semi-parametric models whose parametric components are equivalent to an exponential or multiplicative function. The nonparametric part is estimated on the basis of penalized generalized least squares taking into account smoothness of nonlinear functions. In the empirical study presented market share models with semi-parametric additive brand attractions attain better fits both according to an information criterion that penalizes a model for degrees of freedom (df) consumed and according to error measures determined by bootstrapping.     

Negative norm least‐squares methods for the velocity‐vorticity‐pressure Navier–Stokes equations

We develop and analyze a least‐squares finite element method for the steady state, incompressible Navier–Stokes equations, written as a first‐order system involving vorticity as new dependent variable. In contrast to standard L² least‐squares methods for this system, our approach utilizes discrete negative norms in the least‐squares functional. This allows us to devise efficient preconditioners for the discrete equations, and to establish optimal error estimates under relaxed regularity assumptions. © 1999 John Wiley & Sons, Inc. Numer Methods Partial Differential Eq 15: 237–256, 1999     

生长曲线模型的变量选择

生长曲线模型是一个典型的多元线性模型, 在现代统计学上占有重要地位. 文章首先基于Potthoff-Roy变换后的生长曲线模型, 采用自适应LASSO为惩罚函数给出了参数矩阵的惩罚最小二乘估计, 实现了变量的选择. 其次, 基于局部渐近二次估计, 对生长曲线模型的惩罚最小二乘估计给出了统一的近似估计表达式. 接着, 讨论了经过Potthoff-Roy变换后模型的惩罚最小二乘估计, 证明了自适应LASSO具有Oracle性质. 最后对几种变量选择方法进行了数据模拟. 结果表明自适应LASSO效果比较好. 另外, 综合考虑, Potthoff-Roy变换优于拉直变换.     

Perturbation Analysis of Structured Least Squares Problems and Its Application in Calibration of Interest Rate Term Structure

A structured perturbation analysis of the least squares problem is considered in this paper.The new error bound proves to be sharper than that for general perturbations. We apply the new error bound to study sensitivity of changing the knots for curve fitting of interest rate term structure by cubic spline.Numerical experiments are given to illustrate the sharpness of this bound.     

Local linear regression for data with AR errors

In many statistical applications, data are collected over time, and they are likely correlated. In this paper, we investigate how to incorporate the correlation information into the local linear regression. Under the assumption that the error process is an auto-regressive process, a new estimation procedure is proposed for the nonparametric regression by using local linear regression method and the profile least squares techniques. We further propose the SCAD penalized profile least squares method to determine the order of auto-regressive process. Extensive Monte Carlo simulation studies are conducted to examine the finite sample performance of the proposed procedure, and to compare the performance of the proposed procedures with the existing one. From our empirical studies, the newly proposed procedures can dramatically improve the accuracy of naive local linear regression with working-independent error structure. We illustrate the proposed methodology by an analysis of real data set.     

Penalized regression with individual deviance effects

The present work addresses the problem of model estimation and computations for discrete data when some covariates are modeled smoothly using splines. We propose to introduce and explicitly estimate individual deviance effects (one for each observation), constrained by a ridge penalty. This turns out to be an effective way to absorb model excess variation and detect systematic patterns. Large but very sparse systems of penalized likelihood equations have to be solved. We present fast and compact algorithms for fitting, estimation and computation of the effective dimension. Applications to counts, binomial, and survival data illustrate practical use of this model.     

Likelihood ratio tests for goodness-of-fit of a nonlinear regression model

We propose likelihood and restricted likelihood ratio tests for goodness-of-fit of nonlinear regression. The first-order Taylor approximation around the MLE of the regression parameters is used to approximate the null hypothesis and the alternative is modeled nonparametrically using penalized splines. The exact finite sample distribution of the test statistics is obtained for the linear model approximation and can be easily simulated. We recommend using the restricted likelihood instead of the likelihood ratio test because restricted maximum-likelihood estimates are not as severely biased as the maximum-likelihood estimates in the penalized splines framework.     

Bivariate C 2 cubic spline quasi-interpolants on uniform Powell–Sabin triangulations of a rectangular domain

In this paper we construct discrete quasi-interpolants based on C ² cubic multi-box splines on uniform Powell–Sabin triangulations of a rectangular domain. The main problem consists in finding the coefficient functionals associated with boundary multi-box splines (i.e. multi-box splines whose supports overlap with the domain) involving data points inside or on the boundary of the domain and giving the optimal approximation order. They are obtained either by minimizing an upper bound for the infinity norm of the operator w.r.t. a finite number of free parameters, or by inducing the superconvergence of the gradient of the quasi-interpolant at some specific points of the domain. Finally, we give norm and error estimates and we provide some numerical examples illustrating the approximation properties of the proposed operators.     

On penalty method for equilibrium problems in lexicographic order

In this paper, we consider lexicographic vector equilibrium problems. We propose a penalty function method for solving such problems. We show that every penalty trajectory of the penalized lexicographic equilibrium problem tends to the solution of the original problem. Using the regularized gap function to obtain an error bound result for such penalized problems is given.     

A weighted least squares method for scattered data fitting

In this paper, we present a weighted least squares method to fit scattered data with noise. Existence and uniqueness of a solution are proved and an error bound is derived. The numerical experiments illustrate that our weighted least squares method has better performance than the traditional least squares method in case of noisy data.     

High-Dimensional Menger-Type Curvatures—Part II: d-Separation and a Menagerie of Curvatures

We estimate d-dimensional least squares approximations of an arbitrary d-regular measure μ via discrete curvatures of d+2 variables. The main result bounds the least squares error of approximating μ (or its restrictions to balls) with a d-plane by an average of the discrete Menger-type curvature over a restricted set of simplices. Its proof is constructive and even suggests an algorithm for an approximate least squares d-plane. A consequent result bounds a multiscale error term (used for quantifying the approximation of μ with a sufficiently regular surface) by an integral of the discrete Menger-type curvature over all simplices. The preceding paper (part I) provided the opposite inequalities of these two results. This paper also demonstrates the use of a few other discrete curvatures which are different from the Menger-type curvature. Furthermore, it shows that a curvature suggested by Léger (Ann. Math. 149(3), pp. 831–869, 1999) does not fit within our framework.     

A meshless local Galerkin method for the numerical solution of Hammerstein integral equations based on the moving least squares technique

In this paper, a computational scheme is proposed to estimate the solution of one- and two-dimensional Fredholm-Hammerstein integral equations of the second kind. The method approximates the solution using the discrete Galerkin method based on the moving least squares (MLS) approach as a locally weighted least squares polynomial fitting. The discrete Galerkin technique for integral equations results from the numerical integration of all integrals in the system corresponding to the Galerkin method. Since the proposed method is constructed on a set of scattered points, it does not require any background meshes and so we can call it as the meshless local discrete Galerkin method. The implication of the scheme for solving two-dimensional integral equations is independent of the geometry of the domain. The new method is simple, efficient and more flexible for most classes of nonlinear integral equations. The error analysis of the method is provided. The convergence accuracy of the new technique is tested over several Hammerstein integral equations and obtained results confirm the theoretical error estimates.