Autocovariance Function Estimation via Penalized Regression

The work revisits the autocovariance function estimation, a fundamental problem in statistical inference for time series. We convert the function estimation problem into constrained penalized regression with a generalized penalty that provides us with flexible and accurate estimation, and study the asymptotic properties of the proposed estimator. In case of a nonzero mean time series, we apply a penalized regression technique to a differenced time series, which does not require a separate detrending procedure. In penalized regression, selection of tuning parameters is critical and we propose four different data-driven criteria to determine them. A simulation study shows effectiveness of the tuning parameter selection and that the proposed approach is superior to three existing methods. We also briefly discuss the extension of the proposed approach to interval-valued time series. Supplementary materials for this article are available online.     

生长曲线模型的变量选择

生长曲线模型是一个典型的多元线性模型, 在现代统计学上占有重要地位. 文章首先基于Potthoff-Roy变换后的生长曲线模型, 采用自适应LASSO为惩罚函数给出了参数矩阵的惩罚最小二乘估计, 实现了变量的选择. 其次, 基于局部渐近二次估计, 对生长曲线模型的惩罚最小二乘估计给出了统一的近似估计表达式. 接着, 讨论了经过Potthoff-Roy变换后模型的惩罚最小二乘估计, 证明了自适应LASSO具有Oracle性质. 最后对几种变量选择方法进行了数据模拟. 结果表明自适应LASSO效果比较好. 另外, 综合考虑, Potthoff-Roy变换优于拉直变换.     

Multistage weighted least squares estimation of ARCH processes in the stable and unstable cases

For an ARCH model, we propose a multistage weighted least squares (WLS) estimate which consists of repeated WLS procedures until the corresponding asymptotic variance equals that of the quasi-maximum likelihood estimate (QMLE). At every stage, the current estimate is of a WLS type weighted by the squared conditional variance evaluated at the estimate of the previous stage. Initially, the weighting parameter is any fixed and known value in the parameter space. The procedure provides, without any moment requirement, an asymptotically Gaussian estimate having the same asymptotic distribution as the QMLE even in the unstable case. Apart from the initialization stage, two additional stages are required in the stable case to obtain the same asymptotic distribution as the QMLE, while in the unstable case only one stage is enough. So in all, the proposed procedure involves three stages WLS in the stable case and two stages WLS in the unstable case.     

Variable selection of generalized regression models based on maximum rank correlation

In this paper, we investigate the variable selection problem of the generalized regression models. To estimate the regression parameter, a procedure combining the rank correlation method and the adaptive lasso technique is developed, which is proved to have oracle properties. A modified IMO （iterative marginal optimization） algorithm which directly aims to maximize the penalized rank correlation function is proposed. The effects of the estimating procedure are illustrated by simulation studies.     

SICA for Cox’s proportional hazards model with a diverging number of parameters

The smooth integration of counting and absolute deviation （SICA） penalized variable selection procedure for high-dimensional linear regression models is proposed by Lv and Fan （2009）. In this article, we extend their idea to Cox＇s proportional hazards （PH） model by using a penalized log partial likelihood with the SICA penalty. The number of the regression coefficients is allowed to grow with the sample size. Based on an approximation to the inverse of the Hessian matrix, the proposed method can be easily carried out with the smoothing quasi-Newton （SQN） algorithm. Under appropriate sparsity conditions, we show that the resulting estimator of the regression coefficients possesses the oracle property. We perform an extensive simulation study to compare our approach with other methods and illustrate it on a well known PBC data for predicting survival from risk factors.     

Estimating and Identifying Unspecified Correlation Structure for Longitudinal Data

Identifying correlation structure is important to achieving estimation efficiency in analyzing longitudinal data, and is also crucial for drawing valid statistical inference for large-size clustered data. In this article, we propose a nonparametric method to estimate the correlation structure, which is applicable for discrete longitudinal data. We use eigenvector-based basis matrices to approximate the inverse of the empirical correlation matrix and determine the number of basis matrices via model selection. A penalized objective function based on the difference between the empirical and model approximation of the correlation matrices is adopted to select an informative structure for the correlation matrix. The eigenvector representation of the correlation estimation is capable of reducing the risk of model misspecification, and also provides useful information on the specific within-cluster correlation pattern of the data. We show that the proposed method possesses the oracle property and selects the true correlation structure consistently. The proposed method is illustrated through simulations and two data examples on air pollution and sonar signal studies .     

生长曲线模型的惩罚最小二乘估计

主要考虑了生长曲线模型中的参数矩阵的估计.首先基于Potthoff-Roy变换后的生长曲线模型,采用不同的惩罚函数:Hard Thresholding函数,LASSO,ENET,改进LASSO,SACD给出了参数矩阵的惩罚最小二乘估计.接着对不做变换的生长曲线模型,直接定义其惩罚最小二乘估计,基于Nelder-Mead法给出了估计的数值解算法.最后对提出的参数估计方法进行了数据模拟.结果表明自适应LASSO在估计方面效果比较好.     

Variable selection via generalized SELO-penalized linear regression models

The seamless-L_0(SELO) penalty is a smooth function on [0, ∞) that very closely resembles the L_0 penalty, which has been demonstrated theoretically and practically to be effective in nonconvex penalization for variable selection. In this paper, we first generalize SELO to a class of penalties retaining good features of SELO, and then propose variable selection and estimation in linear models using the proposed generalized SELO(GSELO) penalized least squares(PLS) approach. We show that the GSELO-PLS procedure possesses the oracle property and consistently selects the true model under some regularity conditions in the presence of a diverging number of variables. The entire path of GSELO-PLS estimates can be efficiently computed through a smoothing quasi-Newton(SQN) method. A modified BIC coupled with a continuation strategy is developed to select the optimal tuning parameter. Simulation studies and analysis of a clinical data are carried out to evaluate the finite sample performance of the proposed method. In addition, numerical experiments involving simulation studies and analysis of a microarray data are also conducted for GSELO-PLS in the high-dimensional settings.     

Partial penalized empirical likelihood ratio test under sparse case

A consistent test via the partial penalized empirical likelihood approach for the parametric hypothesis testing under the sparse case, called the partial penalized empirical likelihood ratio (PPELR) test, is proposed in this paper. Our results are demonstrated for the mean vector in multivariate analysis and regression coefficients in linear models, respectively. And we establish its asymptotic distributions under the null hypothesis and the local alternatives of order n^?1/2 under regularity conditions. Meanwhile, the oracle property of the partial penalized empirical likelihood estimator also holds. The proposed PPELR test statistic performs as well as the ordinary empirical likelihood ratio test statistic and outperforms the full penalized empirical likelihood ratio test statistic in term of size and power when the null parameter is zero. Moreover, the proposed method obtains the variable selection as well as the p-values of testing. Numerical simulations and an analysis of Prostate Cancer data confirm our theoretical findings and demonstrate the promising performance of the proposed method in hypothesis testing and variable selection.     

Variable selection and estimation using a continuous approximation to the $$L_0$$ penalty

Variable selection problems are typically addressed under the regularization framework. In this paper, an exponential type penalty which very closely resembles the \(L_0\) penalty is proposed, we called it EXP penalty. The EXP penalized least squares procedure is shown to consistently select the correct model and is asymptotically normal, provided the number of variables grows slower than the number of observations. EXP is efficiently implemented using a coordinate descent algorithm. Furthermore, we propose a modified BIC tuning parameter selection method for EXP and show that it consistently identifies the correct model, while allowing the number of variables to diverge. Simulation results and data example show that the EXP procedure performs very well in a variety of settings.     

Optimal tuning parameter estimation in maximum penalized likelihood method

In maximum penalized or regularized methods, it is important to select a tuning parameter appropriately. This paper proposes a direct plug-in method for tuning parameter selection. The tuning parameters selected using a generalized information criterion (Konishi and Kitagawa, Biometrika, 83, 875–890, 1996) and cross-validation (Stone, Journal of the Royal Statistical Society, Series B, 58, 267–288, 1974) are shown to be asymptotically equivalent to those selected using the proposed method, from the perspective of estimation of an optimal tuning parameter. Because of its directness, the proposed method is superior to the two selection methods mentioned above in terms of computational cost. Some numerical examples which contain the penalized spline generalized linear model regressions are provided.     

Precision Matrix Estimation With ROPE

It is known that the accuracy of the maximum likelihood-based covariance and precision matrix estimates can be improved by penalized log-likelihood estimation. In this article, we propose a ridge-type operator for the precision matrix estimation, ROPE for short, to maximize a penalized likelihood function where the Frobenius norm is used as the penalty function. We show that there is an explicit closed form representation of a shrinkage estimator for the precision matrix when using a penalized log-likelihood, which is analogous to ridge regression in a regression context. The performance of the proposed method is illustrated by a simulation study and real data applications. Computer code used in the example analyses as well as other supplementary materials for this article are available online.     

Generalized Nonparametric Mixed-Effect Models: Computation and Smoothing Parameter Selection

Generalized linear mixed-effect models are widely used for the analysis of correlated non-Gaussian data such as those found in longitudinal studies. In this article, we consider extensions with nonparametric fixed effects and parametric random effects. The estimation is through the penalized likelihood method, and our focus is on the efficient computation and the effective smoothing parameter selection. To assist efficient computation, the joint likelihood of the observations and the latent variables of the random effects is used instead of the marginal likelihood of the observations. For the selection of smoothing parameters and correlation parameters, direct cross-validation techniques are employed; the effectiveness of cross-validation with respect to a few loss functions are evaluated through simulation studies. Real data examples are presented to illustrate potential applications of the methodology. Open-source R code is demonstrated in the Appendix.     

On the impact of model selection on predictor identification and parameter inference

We assessed the ability of several penalized regression methods for linear and logistic models to identify outcome-associated predictors and the impact of predictor selection on parameter inference for practical sample sizes. We studied effect estimates obtained directly from penalized methods (Algorithm 1), or by refitting selected predictors with standard regression (Algorithm 2). For linear models, penalized linear regression, elastic net, smoothly clipped absolute deviation (SCAD), least angle regression and LASSO had a low false negative (FN) predictor selection rates but false positive (FP) rates above 20 % for all sample and effect sizes. Partial least squares regression had few FPs but many FNs. Only relaxo had low FP and FN rates. For logistic models, LASSO and penalized logistic regression had many FPs and few FNs for all sample and effect sizes. SCAD and adaptive logistic regression had low or moderate FP rates but many FNs. 95 % confidence interval coverage of predictors with null effects was approximately 100 % for Algorithm 1 for all methods, and 95 % for Algorithm 2 for large sample and effect sizes. Coverage was low only for penalized partial least squares (linear regression). For outcome-associated predictors, coverage was close to 95 % for Algorithm 2 for large sample and effect sizes for all methods except penalized partial least squares and penalized logistic regression. Coverage was sub-nominal for Algorithm 1. In conclusion, many methods performed comparably, and while Algorithm 2 is preferred to Algorithm 1 for estimation, it yields valid inference only for large effect and sample sizes.     

Adaptive Penalized Weighted Least Absolute Deviations Estimation for the Accelerated Failure Time Model

The accelerated failure time model always offers a valuable complement to the traditional Cox proportional hazards model due to its direct and meaningful interpretation. We propose a variable selection method in the context of the accelerated failure time model for survival data, which can simultaneously complete variable selection and parameter estimation. Meanwhile, the proposed method can deal with the potential outliers in survival times as well as heteroscedastic model errors, which are frequently encountered in practice. Specifically, utilizing the general nonconvex penalty, we propose the adaptive penalized weighted least absolute deviation estimator for the accelerated failure time model. Under some regularity conditions, we show that the proposed method yields consistent estimator and possesses the oracle property. In addition, we propose a new algorithm to compute the estimate in the high dimensional settings, and evaluate the practical utility of the proposed method through extensive simulation studies and two real examples.     

Variable selection for semiparametric varying-coefficient partially linear models with missing response at random

In this paper, we present a variable selection procedure by combining basis function approximations with penalized estimating equations for semiparametric varying-coefficient partially linear models with missing response at random. The proposed procedure simultaneously selects significant variables in parametric components and nonparametric components. With appropriate selection of the tuning parameters, we establish the consistency of the variable selection procedure and the convergence rate of the regularized estimators. A simulation study is undertaken to assess the finite sample performance of the proposed variable selection procedure.     

A Confidence Region Approach to Tuning for Variable Selection

We develop an approach to tuning of penalized regression variable selection methods by calculating the sparsest estimator contained in a confidence region of a specified level. Because confidence intervals/regions are generally understood, tuning penalized regression methods in this way is intuitive and more easily understood by scientists and practitioners. More importantly, our work shows that tuning to a fixed confidence level often performs better than tuning via the common methods based on Akaike information criterion (AIC), Bayesian information criterion (BIC), or cross-validation (CV) over a wide range of sample sizes and levels of sparsity. Additionally, we prove that by tuning with a sequence of confidence levels converging to one, asymptotic selection consistency is obtained, and with a simple two-stage procedure, an oracle property is achieved. The confidence-region-based tuning parameter is easily calculated using output from existing penalized regression computer packages. Our work also shows how to map any penalty parameter to a corresponding confidence coefficient. This mapping facilitates comparisons of tuning parameter selection methods such as AIC, BIC, and CV, and reveals that the resulting tuning parameters correspond to confidence levels that are extremely low, and can vary greatly across datasets. Supplemental materials for the article are available online.     

A Penalized Likelihood Approach to Rotation of Principal Components

A new paradigm for enhancing the interpretability of principal components through rotation is presented within the framework of penalized likelihood. The rotated components are computed as the maximizers of a Gaussian-based profile log-likelihood function plus a penalty term defined by a standard rotation criterion. This method enjoys a number of advantages over other methods for principal component rotation, notably (1) the rotation specifically targets ill-defined principal components, which may benefit the most from rotation, and (2) the connection with likelihood allows assessment of the fidelity of the rotated components to the data, thereby guiding the choice of penalty parameter. The method is illustrated with an application to a small functional dataset. Efficient computation of the penalized likelihood solution is possible using recently developed algorithms for optimization under orthogonality constraints.     

Hierarchically penalized additive hazards model with diverging number of parameters

In many applications,covariates can be naturally grouped.For example,for gene expression data analysis,genes belonging to the same pathway might be viewed as a group.This paper studies variable selection problem for censored survival data in the additive hazards model when covariates are grouped.A hierarchical regularization method is proposed to simultaneously estimate parameters and select important variables at both the group level and the within-group level.For the situations in which the number of parameters tends to∞as the sample size increases,we establish an oracle property and asymptotic normality property of the proposed estimators.Numerical results indicate that the hierarchically penalized method performs better than some existing methods such as lasso,smoothly clipped absolute deviation(SCAD)and adaptive lasso.     

响应变量随机缺失下变系数模型的变量选择

In this paper,we present a variable selection procedure by combining basis function approximations with penalized estimating equations for varying-coefficient models with missing response at random.With appropriate selection of the tuning parameters,we establish the consistency of the variable selection procedure and the optimal convergence rate of the regularized estimators.A simulation study is undertaken to assess the finite sample performance of the proposed variable selection procedure.