岭回归分析的SAS程序设计

田俊．岭回归分析的ＳＡＳ程序设计．岭回归分析方法是传统的多元回归分析方法的一个补充,在实际工作中经常使用。但是在标准统计软件ＳＡＳ中没有专门的岭回归分析过程,本文介绍如何通过设置伪样品后使用ＳＡＳ进行岭回归分析     

岭回归和核主成分回归在消除共线性中的实证分析及比较

检测和解决多元回归分析中的多重共线性问题具有重要意义.本文采用岭回归(RR)和核主成分回归(KCPR)对同一数据进行回归分析,使用方差膨胀因子(VIF)和条件指数(CI)作为共线性诊断的量度,并对回归模型结果进行比较.经过实证分析,发现这两种回归方法都能很好地消除多重共线性,总的来说核主成分回归的对内拟合效果要优于岭回归.但是这两种方法的参数选择的不同对回归模型的好坏都有巨大影响,需要进一步分析判断.     

基于岭回归模型大数据最优子抽样算法研究

随着大数据时代的来临,为了提高计算效率,Wang等(2018)提出基于logistic回归的最优子抽样算法,在保证参数估计精度的前提下,节省了大量的运算时间.为解决变量间的多重共线性,文章提出基于岭回归模型的最优子抽样算法,并证明岭回归模型中参数估计的一致性与渐近正态性.利用数值模拟与实证分析对最优子抽样算法进行评估,...     

基于多重共线性的处理方法

多重共线性简称共线性是多元线性回归分析中一个重要问题。消除共线性的危害一直是回归分析的一个重点。目前处理严重共线性的常用方法有以下几种:岭回归、主成分回归、逐步回归、偏最小二乘法、Lasso回归等。本文就这几种方法进行比较分析,介绍它们的优缺点,通过实例分析以便于选择合适的方法处理共线性。     

半参数回归模型的几乎无偏岭估计

提出了半参数回归模型的几乎无偏岭估计,并与岭估计进行了比较,在均方误差意义下,几乎无偏岭估计优于岭估计. 然后讨论了有偏参数的选取问题. 最后,用模拟算例和实际应用说明了几乎无偏岭估计的有效性和可行性.     

求解岭回归问题的加速增广投影算法

正1引言考虑求解岭回归或者Tikhonov正则化最小二乘回归问题■这里X是一个m×n的复矩阵,β是一个n维未知向量,y是一个m维的复向量,λ是正则化参数,‖·‖2表示向量的欧拉范数.岭回归问题对病态数据的拟合效果要强于最小二乘法.目前,岭回归问题已广泛应用于数据分析、机器学习、电网等领域.近年来,一系列随机算法被用来求解大规模线性系统.Strohmer和Vershynin [1]提出     

偏最小二乘回归在选矿效益建模中的应用

选矿效益对矿管理有着重要意义。影响选矿效益的因素之间存在严重的多重共线性,采用PLS方法建立了选矿效益的偏最小二乘回归模型,避免了普通多元回归模型的不合理性和岭回归模型选择岭参数的主观性。     

PPR的收敛性和全面攻击导弹数据处理

本文证明了投影寻踪回归(Projection Pursuit Regression简称PPR)中岭函数为多项式形式时,PPR的L_2收敛性;对岭函数为多项式形式的投影寻踪回归给出了一种新的算法;应用这种算法对全向攻击导弹数据进行了处理,所获得的回归模型可供科研单位实际使用.     

基于泛岭估计对岭估计过度压缩的改进方法

岭估计是解决多元线性回归多重共线性问题的有效方法,是有偏的压缩估计。与普通最小二乘估计相比,岭估计可以降低参数估计的均方误差,但是却增大残差平方和,拟合效果变差。本文提出一种基于泛岭估计对岭估计过度压缩的改进方法,可以改进岭估计的拟合效果,减小岭估计残差平方和的增加幅度。     

高维线性回归模型中非凸惩罚岭回归

非凸惩罚函数包括SCAD惩罚和MCP惩罚, 这类惩罚函数具有无偏性、连续性和稀疏性等特点,岭回归方法能够很好的克服共线性问题. 本文将非凸惩罚函数和岭回归方法的优势结合起来(简记为 NPR),研究了自变量间存在高相关性问题时NPR估计的Oracle性质. 这里主要研究了参数个数$p_n$ 随样本量$n$ 呈指数阶增长的情况. 同时, 通过模拟研究和实例分析进一步验证了NPR 方法的表现.     

Regularized fuzzy clusterwise ridge regression

Fuzzy clusterwise regression has been a useful method for investigating cluster-level heterogeneity of observations based on linear regression. This method integrates fuzzy clustering and ordinary least-squares regression, thereby enabling to estimate regression coefficients for each cluster and fuzzy cluster memberships of observations simultaneously. In practice, however, fuzzy clusterwise regression may suffer from multicollinearity as it builds on ordinary least-squares regression. To deal with this problem in fuzzy clusterwise regression, a new method, called regularized fuzzy clusterwise ridge regression, is proposed that combines ridge regression with regularized fuzzy clustering in a unified framework. In the proposed method, ridge regression is adopted to estimate clusterwise regression coefficients while handling potential multicollinearity among predictor variables. In addition, regularized fuzzy clustering based on maximizing entropy is utilized to systematically determine an optimal degree of fuzziness in memberships. A simulation study is conducted to evaluate parameter recovery of the proposed method as compared to the extant non-regularized counterpart. The usefulness of the proposed method is illustrated by an application concerning the relationship among the characteristics of used cars.     

多元协方差阵扰动模型岭估计的影响分析

本文研究了多元线性同归模型岭估计的影响分析问题.利用最小二乘估计方法,获得了多元协方差阵扰动模型与原模型参数阵之间的岭估计的一些关系式,给出了度量影响大小的基于岭估计的广义Cook距离.     

Robust kernel ridge regression based on M-estimation

Ridge regression (RR) and kernel ridge regression (KRR) are important tools to avoid the effects of multicollinearity. However, the predictions of RR and KRR become inappropriate for use in regression models when data are contaminated by outliers. In this paper, we propose an algorithm to obtain a nonlinear robust prediction without specifying a nonlinear model in advance. We combine M-estimation and kernel ridge regression to obtain the nonlinear prediction. Then, we compare the proposed method with some other methods.     

A stochastic restricted ridge regression estimator

Groß [J. Groß, Restricted ridge estimation, Statistics & Probability Letters 65 (2003) 57–64] proposed a restricted ridge regression estimator when exact restrictions are assumed to hold. When there are stochastic linear restrictions on the parameter vector, we introduce a new estimator by combining ideas underlying the mixed and the ridge regression estimators under the assumption that the errors are not independent and identically distributed. Apart from [J. Groß, Restricted ridge estimation, Statistics & Probability Letters 65 (2003) 57–64], we call this new estimator as the stochastic restricted ridge regression (SRRR) estimator. The performance of the SRRR estimator over the mixed estimator in respect of the variance and the mean square error matrices is examined. We also illustrate our findings with a numerical example. The shrinkage generalized least squares (GLS) and the stochastic restricted shrinkage GLS estimators are proposed.     

多元广义岭估计及K值选取的 Q(c)准则

当自变量间存在复共线性时,最小二乘估计就表现出不稳定并可能导致错误的结果.本文采用广义岭估计β(K)来估计多元线性模型的回归系数β=vec(B),通过岭参数K值的选取,可使广义岭估计的均方误差MSE小于最小二乘估计的MSE.指出了广义岭估计中根据MSE准则选取K值存在的主要缺陷,采用了一种选取K值的新准则Q(c),它包含MSE准则和最小二乘LS准则作为特例,从理论上证明和讨论了Q(c)准则的优良性,阐明了c值的统计含义,并给出了确定c值的方法.     

Improved ridge regression estimators for the logistic regression model

The estimation of the regression parameters for the ill-conditioned logistic regression model is considered in this paper. We proposed five ridge regression (RR) estimators, namely, unrestricted RR, restricted ridge regression, preliminary test RR, shrinkage ridge regression and positive rule RR estimators for estimating the parameters $(\beta )$ when it is suspected that the parameter $\beta $ may belong to a linear subspace defined by $H\beta =h$ . Asymptotic properties of the estimators are studied with respect to quadratic risks. The performances of the proposed estimators are compared based on the quadratic bias and risk functions under both null and alternative hypotheses, which specify certain restrictions on the regression parameters. The conditions of superiority of the proposed estimators for departure and ridge parameters are given. Some graphical representations and efficiency analysis have been presented which support the findings of the paper.     

Penalized Nonparametric Scalar-on-Function Regression via Principal Coordinates

A number of classical approaches to nonparametric regression have recently been extended to the case of functional predictors. This article introduces a new method of this type, which extends intermediate-rank penalized smoothing to scalar-on-function regression. In the proposed method, which we call principal coordinate ridge regression, one regresses the response on leading principal coordinates defined by a relevant distance among the functional predictors, while applying a ridge penalty. Our publicly available implementation, based on generalized additive modeling software, allows for fast optimal tuning parameter selection and for extensions to multiple functional predictors, exponential family-valued responses, and mixed-effects models. In an application to signature verification data, principal coordinate ridge regression, with dynamic time warping distance used to define the principal coordinates, is shown to outperform a functional generalized linear model. Supplementary materials for this article are available online.     

基于奇异值分解的岭型回归(英文)

本文基于设计阵的奇异值分解(SVD),从LS估计出发,应用岭回归估计方法,构造了回归系数的一个新的有偏估计,称为基于SVD的岭型回归估计,简称RRSVD估计,讨论了其性质和偏参数的选取问题,得到了许多重要结论.计算结果表明,在设计阵呈病态时,RRS善岭回归估计.