Shrinkage estimation for identification of linear components in additive models

In this short paper, we demonstrate that the popular penalized estimation method typically used for variable selection in parametric or semiparametric models can actually provide a way to identify linear components in additive models. Unlike most studies in the literature, we are NOT performing variable selection. Due to the difficulty in a priori deciding which predictors should enter the partially linear additive model as the linear components, such a method will prove useful in practice.     

Sparse Partially Linear Additive Models

The generalized partially linear additive model (GPLAM) is a flexible and interpretable approach to building predictive models. It combines features in an additive manner, allowing each to have either a linear or nonlinear effect on the response. However, the choice of which features to treat as linear or nonlinear is typically assumed known. Thus, to make a GPLAM a viable approach in situations in which little is known a priori about the features, one must overcome two primary model selection challenges: deciding which features to include in the model and determining which of these features to treat nonlinearly. We introduce the sparse partially linear additive model (SPLAM), which combines model fitting and both of these model selection challenges into a single convex optimization problem. SPLAM provides a bridge between the lasso and sparse additive models. Through a statistical oracle inequality and thorough simulation, we demonstrate that SPLAM can outperform other methods across a broad spectrum of statistical regimes, including the high-dimensional (p ? N) setting. We develop efficient algorithms that are applied to real datasets with half a million samples and over 45,000 features with excellent predictive performance. Supplementary materials for this article are available online.     

固定效应部分线性可加面板数据模型的惩罚二次推断估计

具有较强解释力和灵活性的部分线性可加面板数据模型在各学科领域应用广泛.针对个体内存在相关结构的固定效应部分线性可加面板数据模型,本文在结合幂样条函数和最小二乘虚拟变量(LSDV)法的基础上,利用惩罚二次推断函数(PQIF)法对模型进行估计,在一定的正则条件下,证明了参数估计的渐近正态性和非参数估计的收敛性,Monte Carlo数值模拟显示所述估计方法具有良好的有限样本表现,同时,我们还将估计技术应用于实际数据分析中.     

因变量缺失下部分线性变系数变量含误差模型的估计

该文主要考虑部分线性变系数模型在自变量含有测量误差以及因变量存在缺失情形下的估计问题.基于Profile最小二乘技术,针对参数分量和非参数分量提出了多种估计方法.第一种估计方法只利用了完整观测数据,而第二种和第三种估计方法分别利用了插补技术和替代技术.参数分量的所有估计被证明是渐近正态的,非参数分量的所有估计被证明和一般非参数回归函数的估计具有相同的收敛速度.对于因变量的均值,构造了两类估计并证明了它们的渐近正态性.最后,通过数值模拟验证了所提方法.     

Generalized Partially Linear Measurement Error Models

This article considers generalized partially linear models when the linear covariate is measured with additive error. We propose estimators of parameter and nonparametric function by using local linear regression, the SIMEX technique, and generalized estimating equation. The asymptotic normality of the estimators of the parameter, and bias and variance of the estimators of the nonparametric component are derived under appropriate assumptions. In addition, the generalization to clustered measurements is discussed. The approaches are used to the analysis of data from the Framingham Heart Study. A simulation experiment is conducted for an illustration.     

�������Բ������ģ���еĹ�������

In this paper, we consider the estimation problem for partially linear models with additive measurement errors in the nonparametric part. Two kinds of estimators are proposed. The first one is an integral moment-based estimator with deconvolution kernel techniques, associated with the strong consistency for the estimator. Another one is a simulation-based estimator to avoid the integrals involved in the integral moment-based estimator. Simulation studies are conducted to examine the performance of the proposed estimators.     

纵向数据部分线性测量误差模型的二次推断函数估计

基于纵向数据部分线性测量误差模型, 研究了模型中兴趣参数部分回归系数的估计问题. 首先采用B样条方法逼近模型中的非参数函数, 然后提出修正的二次推断函数(QIF)方法对模型中参数部分的回归系数进行估计, 所提方法可以提高估计的效率. 在一定的正则条件下, 证明了所得到的估计量具有相合性和渐近正态性. 最后, 通过模拟研究和实例分析验证了所提出估计方法的有限大样本性质.     

Difference based estimation for partially linear regression models with measurement errors

This paper is concerned with the estimating problem of the partially linear regression models where the linear covariates are measured with additive errors. A difference based estimation is proposed to estimate the parametric component. We show that the resulting estimator is asymptotically unbiased and achieves the semiparametric efficiency bound if the order of the difference tends to infinity. The asymptotic normality of the resulting estimator is established as well. Compared with the corrected profile least squares estimation, the proposed procedure avoids the bandwidth selection. In addition, the difference based estimation of the error variance is also considered. For the nonparametric component, the local polynomial technique is implemented. The finite sample properties of the developed methodology is investigated through simulation studies. An example of application is also illustrated.     

Functional Additive Mixed Models

We propose an extensive framework for additive regression models for correlated functional responses, allowing for multiple partially nested or crossed functional random effects with flexible correlation structures for, for example, spatial, temporal, or longitudinal functional data. Additionally, our framework includes linear and nonlinear effects of functional and scalar covariates that may vary smoothly over the index of the functional response. It accommodates densely or sparsely observed functional responses and predictors which may be observed with additional error and includes both spline-based and functional principal component-based terms. Estimation and inference in this framework is based on standard additive mixed models, allowing us to take advantage of established methods and robust, flexible algorithms. We provide easy-to-use open source software in the pffr() function for the R package refund. Simulations show that the proposed method recovers relevant effects reliably, handles small sample sizes well, and also scales to larger datasets. Applications with spatially and longitudinally observed functional data demonstrate the flexibility in modeling and interpretability of results of our approach.     

Partially varying coefficient single-index additive hazard models

The partially linear additive hazards model has been proposed to study the interaction between some covariates and an exposure variable. In this paper, we extend it to the partially varying coefficient single-index additive hazard model where the high dimension covariates are collapsed to a single index, due to practical needs. Two sets of estimating equations were proposed to estimate the varying coefficient functions in the linear components: the link function for the single index and the single-index parameter vector separately. It was shown that the proposed local and global estimators are asymptotically normal. Simulation studies were conducted to examine the finite-sample performance of our method to compare the relative performance of our method with existing ones. A real data analysis was used to illustrate the proposed methods.     

Note on “Some models for deriving the priority weights from interval fuzzy preference relations”

Deriving accurate interval weights from interval fuzzy preference relations is key to successfully solving decision making problems. Xu and Chen (2008) proposed a number of linear programming models to derive interval weights, but the definitions for the additive consistent interval fuzzy preference relation and the linear programming model still need to be improved. In this paper, a numerical example is given to show how these definitions and models can be improved to increase accuracy. A new additive consistency definition for interval fuzzy preference relations is proposed and novel linear programming models are established to demonstrate the generation of interval weights from an interval fuzzy preference relation.     

半相依部分线性可加回归模型的统计推断

受实际问题研究的启发, 为减少模型偏差, 提出了一类半相依部分线性可加的半参数回归模型. 这类半相依模型中, 响应变量与 一部分解释变量之间的关系是线性的, 与另一部分解释变量之间的关系未知但具有可加结构, 各方程的误差之间是相关的. 将级 数逼近法、最小二乘法和同期相关的估计结合起来, 提出了用于估计模型参数分量的加权半参数最小二乘估计量(WSLSEs), 和用于估 计模型非参数分量的加权级数逼近估计量(WSEs). 证明了这些加权的估计量比相应的不加权的估计量渐近有效, 并导出了相应的渐近正态性. 另外, 还讨论了利用这些估计量的渐近性质来对模型的参数及非参数分量作统计推断. 用大量的模拟实验考察 了所提出的方法在有限样本情况下的表现, 并对美国的一个关于妇女工资问题的全国纵向调查(NLS)数据集进行了统计分析.     

Position of Principal component analysis among Auto-associative composite models

We introduce Auto-associative composite models, which have shown a good behavior on real data sets, and share important theoretical approximation properties. Their basic principle is to approximate iteratively data by manifolds of increasing dimension. We exhibit a special class of such models: auto-associative additive models. Their use is widespread in Projection pursuit regression. First, we show that Principal component analysis is a linear auto-associative additive model. Then, we show that principal component analysis is the only auto-associative composite model which is additive.     

Semiparametric and Additive Model Selection Using an Improved Akaike Information Criterion

Abstract

An improved AIC-based criterion is derived for model selection in general smoothing-based modeling, including semiparametric models and additive models. Examples are provided of applications to goodness-of-fit, smoothing parameter and variable selection in an additive model and semiparametric models, and variable selection in a model with a nonlinear function of linear terms.     

带限制线性混合模型中参数估计的小样本性质

本文将带有线性限制下的线性模型理论推广至带有一般线性限制下的线性混合效应模型.同时,本文在没有李(2010)中的正则条件下,构造了估计,考虑了估计的小样本性质.     

部分线性变系数模型Backfitting估计的渐近性质

作为部分线性模型与变系数模型的推广,部分线性变系数模型是一类应用广泛的数据分析模型.利用Backfitting方法拟合这类特殊的可加模型,可得到模型中常值系数估计量的精确解析表达式,该估计量被证明是n~(1/2)相合的.最后通过数值模拟考察了所提估计方法的有效性.     

Model identification and selection for single-index varying-coefficient models

Annals of the Institute of Statistical Mathematics - Single-index varying-coefficient models include many types of popular semiparametric models, i.e., single-index models, partially linear models,...     

Bayesian hierarchical linear mixed models for additive smoothing splines

Bayesian hierarchical models have been used for smoothing splines, thin-plate splines, and L-splines. In analyzing high dimensional data sets, additive models and backfitting methods are often used. A full Bayesian analysis for such models may include a large number of random effects, many of which are not intuitive, so researchers typically use noninformative improper or nearly improper priors. We investigate propriety of the posterior for these cases. Our findings extend known results for normal linear mixed models to certain cases with Bayesian additive smoothing spline models. Supported by National Science Foundation grant SES-0351523 and by National Institutes of Health grants R01-CA100760 and R01-MH071418.     

Semiparametric Approximation Methods in Multivariate Model Selection

In this paper we propose a cross-validation selection criterion to determine asymptotically the correct model among the family of all possible partially linear models when the underlying model is a partially linear model. We establish the asymptotic consistency of the criterion. In addition, the criterion is illustrated using two real sets of data.     

Variable Selection for Partially Linear Models via Adaptive LASSO

Partially linear model is a class of commonly used semiparametric models, this paper focus on variable selection and parameter estimation for partially linear models via adaptive LASSO method. Firstly, based on profile least squares and adaptive LASSO method, the adaptive LASSO estimator for partially linear models are constructed, and the selections of penalty parameter and bandwidth are discussed. Under some regular conditions, the consistency and asymptotic normality for the estimator are investigated, and it is proved that the adaptive LASSO estimator has the oracle properties. The proposed method can be easily implemented. Finally a Monte Carlo simulation study is conducted to assess the finite sample performance of the proposed variable selection procedure, results show the adaptive LASSO estimator behaves well.