基于奇异值分解的岭型回归(英文)

本文基于设计阵的奇异值分解(SVD),从LS估计出发,应用岭回归估计方法,构造了回归系数的一个新的有偏估计,称为基于SVD的岭型回归估计,简称RRSVD估计,讨论了其性质和偏参数的选取问题,得到了许多重要结论.计算结果表明,在设计阵呈病态时,RRS善岭回归估计.     

Generalized Isotonic Regression

We present a new computational and statistical approach for fitting isotonic models under convex differentiable loss functions through recursive partitioning. Models along the partitioning path are also isotonic and can be viewed as regularized solutions to the problem. Our approach generalizes and subsumes the well-known work of Barlow and Brunk on fitting isotonic regressions subject to specially structured loss functions, and expands the range of loss functions that can be used (e.g., adding Huber loss for robust regression). This is accomplished through an algorithmic adjustment to a recursive partitioning approach recently developed for solving large-scale ?₂-loss isotonic regression problems. We prove that the new algorithm solves the generalized problem while maintaining the favorable computational and statistical properties of the l₂ algorithm. The results are demonstrated on both real and synthetic data in two settings: fitting count data using negative Poisson log-likelihood loss, and fitting robust isotonic regressions using Huber loss. Proofs of theorems and a MATLAB-based software package implementing our algorithm are available in the online supplementary materials.     

An efficient method to set up a Lanczos based preconditioner for discrete ill-posed problems

Rezghi and Hosseini [M. Rezghi, S.M. Hosseini, Lanczos based preconditioner for discrete ill-posed problems, Computing 88 (2010) 79–96] presented a Lanczos based preconditioner for discrete ill-posed problems. Their preconditioner is constructed by using few steps (e.g., k) of the Lanczos bidiagonalization and corresponding computed singular values and right Lanczos vectors. In this article, we propose an efficient method to set up such preconditioner. Some numerical examples are given to show the effectiveness of the method.     

Lower Rank Approximation of Matrices Based on Fast and Robust Alternating Regression

We introduce fast and robust algorithms for lower rank approximation to given matrices based on robust alternating regression. The alternating least squares regression, also called criss-cross regression, was used for lower rank approximation of matrices, but it lacks robustness against outliers in these matrices. We use robust regression estimators and address some of the complications arising from this approach. We find it helpful to use high breakdown estimators in the initial iterations, followed by M estimators with monotone score functions in later iterations towards convergence. In addition to robustness, the computational speed is another important consideration in the development of our proposed algorithm, because alternating robust regression can be computationally intensive for large matrices. Based on a mix of the least trimmed squares (LTS) and Huber's M estimators, we demonstrate that fast and robust lower rank approximations are possible for modestly large matrices.     

The Generalized Ridge Trace Plot: Visualizing Bias and Precision

In ridge regression and related shrinkage methods, the ridge trace plot, a plot of estimated coefficients against a shrinkage parameter, is a common graphical adjunct to help determine a favorable trade-off of bias against precision (inverse variance) of the estimates. However, standard unidimensional versions of this plot are ill-suited for this purpose because they show only bias directly and ignore the multidimensional nature of the problem.

A generalized version of the ridge trace plot is introduced, showing covariance ellipsoids in parameter space, whose centers show bias and whose size and shape show variance and covariance, respectively, in relation to the criteria for which these methods were developed. These provide a direct visualization of both bias and precision. Even two-dimensional bivariate versions of this plot show interesting features not revealed in the standard univariate version. Low-rank versions of this plot, based on an orthogonal transformation of predictor space extend these ideas to larger numbers of predictor variables, by focusing on the dimensions in the space of predictors that are likely to be most informative about the nature of bias and precision. Two well-known datasets are used to illustrate these graphical methods. The genridge package for R implements computation and display.     

Semiparametric Bayesian Regression via Potts Model

We consider Bayesian nonparametric regression through random partition models. Our approach involves the construction of a covariate-dependent prior distribution on partitions of individuals. Our goal is to use covariate information to improve predictive inference. To do so, we propose a prior on partitions based on the Potts clustering model associated with the observed covariates. This drives by covariate proximity both the formation of clusters, and the prior predictive distribution. The resulting prior model is flexible enough to support many different types of likelihood models. We focus the discussion on nonparametric regression. Implementation details are discussed for the specific case of multivariate multiple linear regression. The proposed model performs well in terms of model fitting and prediction when compared to other alternative nonparametric regression approaches. We illustrate the methodology with an application to the health status of nations at the turn of the 21st century. Supplementary materials are available online.     

A Note on Sin Θ Theorems for Singular Subspace Variations

New perturbation theorems for bases of singular subspaces are proved. These theorems complement the known sin theorems for singular subspace perturbations, taking into account a kind of sensitivity of singular vectors discarded by previous theorems. Furthermore these results guarantee that high relative accuracy algorithms for the SVD are able to compute reliably simultaneous bases of left and right singular subspaces.     

线性回归模型中方差变点的连续监测

The paper investigates the sequential observations’ variance change in linear regression model. The procedure is based on a detection function constructed by residual squares of CUSUM and a boundary function which is designed so that the test has a small probability of false alarm and asymptotic power one. Simulation results show our monitoring procedure performs well when variance change occurs shortly after the monitoring time. The method is still feasible for regression coefficients change or both variance and regression coefficients change problem.     

累积Logistic回归模型结构变点的序贯检验

分类时间序列在生物医学、社会学和遗传学等领域有着广泛的应用,累积Logistic回归模型是分类时间序列建模的一类重要模型.本文基于偏似然得分过程(Partial likelihood score process)提出一种变点序贯检验方法,监测累积Logistic回归模型的结构是否发生变化.原假设下推导检验统计量的极限分...     

Solution of a class of singular boundary value problems

In this work a class of singular ordinary differential equations is considered. These problems arise from many engineering and physics applications such as electro-hydrodynamics and some thermal explosions. Adomian decomposition method is applied to solve these singular boundary value problems. The approximate solution is calculated in the form of series with easily computable components. The method is tested for its efficiency by considering four examples and results are compared with previous known results. Techniques that can be applied to obtain higher accuracy of the present method has also been discussed.     

A singular boundary value problem for odd-order differential equations

The odd-order differential equation (−1)ⁿx⁽²ⁿ⁺¹⁾=f(t,x,…,x⁽²ⁿ⁾) together with the Lidstone boundary conditions x^(2j)(0)=x^(2j)(T)=0, 0?j?n−1, and the next condition x⁽²ⁿ⁾(0)=0 is discussed. Here f satisfying the local Carathéodory conditions can have singularities at the value zero of all its phase variables. Existence result for the above problem is proved by the general existence principle for singular boundary value problems.     

Analysis of Support Vector Machines Regression

Support vector machines regression (SVMR) is a regularized learning algorithm in reproducing kernel Hilbert spaces with a loss function called the ε-insensitive loss function. Compared with the well-understood least square regression, the study of SVMR is not satisfactory, especially the quantitative estimates of the convergence of this algorithm. This paper provides an error analysis for SVMR, and introduces some recently developed methods for analysis of classification algorithms such as the projection operator and the iteration technique. The main result is an explicit learning rate for the SVMR algorithm under some assumptions. Research supported by NNSF of China No. 10471002, No. 10571010 and RFDP of China No. 20060001010.     

Estimating a reduced rank regression model for non-normal variables

A method using third order moments for estimating the regression coefficients as well as the latent state scores of the reduced-rank regression model when the latent variable(s) are non-normally distributed is presented in this paper. It is shown that the factor analysis type indeterminacy of the regression coefficient matrices is eliminated. A real life example of the proposed method is presented. Differences of this solution with the reduced-rank regression eigen solution are discussed.     

On Extreme Regression Quantiles

Smith (1994) introduced the idea of extreme regression quantiles and he developed some asymptotic results for algebraically tailed error distributions. The results provided a close analogy to standard extreme value theory for one-sample extremes. Here we obtain the following generalizations. First, an extreme value distribution theory is developed in the exponentially tailed case, where the extreme slope estimates need not diverge to infinity and may actually be consistent. The design conditions of Smith (1994) are also generalized. Second, the tail behavior measure of Jureckova´ (1981) and He et al. (1990) is considered for extreme quantiles. For algebraically tailed error distributions, the average right extreme regression fit acts like a one-sample right extreme; while in the exponentially tailed case, the tail behavior is more like that of a slightly more central order statistic.     

Hazard Rate Regression Using Ordinary Nonparametric Regression Smoothers

Abstract

This article proposes a method for nonparametric estimation of hazard rates as a function of time and possibly multiple covariates. The method is based on dividing the time axis into intervals, and calculating number of event and follow-up time contributions from the different intervals. The number of event and follow-up time data are then separately smoothed on time and the covariates, and the hazard rate estimators obtained by taking the ratio. Pointwise consistency and asymptotic normality are shown for the hazard rate estimators for a certain class of smoothers, which includes some standard approaches to locally weighted regression and kernel regression. It is shown through simulation that a variance estimator based on this asymptotic distribution is reasonably reliable in practice. The problem of how to select the smoothing parameter is considered, but a satisfactory resolution to this problem has not been identified. The method is illustrated using data from several breast cancer clinical trials.     

Regression Models for Multivariate Count Data

Data with multivariate count responses frequently occur in modern applications. The commonly used multinomial-logit model is limiting due to its restrictive mean-variance structure. For instance, analyzing count data from the recent RNA-seq technology by the multinomial-logit model leads to serious errors in hypothesis testing. The ubiquity of overdispersion and complicated correlation structures among multivariate counts calls for more flexible regression models. In this article, we study some generalized linear models that incorporate various correlation structures among the counts. Current literature lacks a treatment of these models, partly because they do not belong to the natural exponential family. We study the estimation, testing, and variable selection for these models in a unifying framework. The regression models are compared on both synthetic and real RNA-seq data. Supplementary materials for this article are available online.     

The Catline for Deep Regression

Motivated by the notion of regression depth (Rousseeuw and Hubert, 1996) we introduce thecatline, a new method for simple linear regression. At any bivariate data setZ_n={(x_i, y_i);i=1, …, n} its regression depth is at leastn/3. This lower bound is attained for data lying on a convex or concave curve, whereas for perfectly linear data the catline attains a depth ofn. We construct anO(n log n) algorithm for the catline, so it can be computed fast in practice. The catline is Fisher-consistent at any linear modely=βx+α+ein which the error distribution satisfies med(e | x)=0, which encompasses skewed and/or heteroscedastic errors. The breakdown value of the catline is 1/3, and its influence function is bounded. At the bivariate gaussian distribution its asymptotic relative efficiency compared to theL¹line is 79.3% for the slope, and 88.9% for the intercept. The finite-sample relative efficiencies are in close agreement with these values. This combination of properties makes the catline an attractive fitting method.