A Parallel Algorithm for Large-Scale Nonconvex Penalized Quantile Regression

Penalized quantile regression (PQR) provides a useful tool for analyzing high-dimensional data with heterogeneity. However, its computation is challenging due to the nonsmoothness and (sometimes) the nonconvexity of the objective function. An iterative coordinate descent algorithm (QICD) was recently proposed to solve PQR with nonconvex penalty. The QICD significantly improves the computational speed but requires a double-loop. In this article, we propose an alternative algorithm based on the alternating direction method of multiplier (ADMM). By writing the PQR into a special ADMM form, we can solve the iterations exactly without using coordinate descent. This results in a new single-loop algorithm, which we refer to as the QPADM algorithm. The QPADM demonstrates favorable performance in both computational speed and statistical accuracy, particularly when the sample size n and/or the number of features p are large. Supplementary material for this article is available online.     

A Finite Smoothing Algorithm for Quantile Regression

This article introduces a new method for computing regression quantile functions. This method applies a finite smoothing algorithm based on smoothing the nondifferentiable quantile regression objective function ρ_τ. The smoothing can be done for all τ ∈ (0, 1), and the convergence is finite for any finite number of τ_i ∈ (0, 1), i = 1,…,N. Numerical comparison shows that the finite smoothing algorithm outperforms the simplex algorithm in computing speed. Compared with the powerful interior point algorithm, which was introduced in an earlier article, it is competitive overall; however, it is significantly faster than the interior point algorithm when the design matrix in quantile regression has a large number of covariates. Additionally, the new algorithm provides the same accuracy as the simplex algorithm. In contrast, the interior point algorithm gives only the approximate solutions in theory, and rounding may be necessary to improve the accuracy of these solutions in practice.     

Efficient Quantile Estimation for Functional-Coefficient Partially Linear Regression Models

The quantile estimation methods are proposed for functional-coefficient partially linear regression (FCPLR) model by combining nonparametric and functional-coefficient regression (FCR) model. The local linear scheme and the integrated method are used to obtain local quantile estimators of all unknown functions in the FCPLR model. These resulting estimators are asymptotically normal, but each of them has big variance. To reduce variances of these quantile estimators, the one-step backfitting technique is used to obtain the efficient quantile estimators of all unknown functions, and their asymptotic normalities are derived. Two simulated examples are carried out to illustrate the proposed estimation methodology.     

Computational Experience with a Safeguarded Barrier Algorithm for Sparse Nonlinear Programming

We describe an enhanced version of the primal-dual interior point algorithm in Lasdon, Plummer, and Yu (ORSA Journal on Computing, vol. 7, no. 3, pp. 321–332, 1995), designed to improve convergence with minimal loss of efficiency, and designed to solve large sparse nonlinear problems which may not be convex. New features include (a) a backtracking linesearch using an L1 exact penalty function, (b) ensuring that search directions are downhill for this function by increasing Lagrangian Hessian diagonal elements when necessary, (c) a quasi-Newton option, where the Lagrangian Hessian is replaced by a positive definite approximation (d) inexact solution of each barrier subproblem, in order to approach the central trajectory as the barrier parameter approaches zero, and (e) solution of the symmetric indefinite linear Newton equations using a multifrontal sparse Gaussian elimination procedure, as implemented in the MA47 subroutine from the Harwell Library (Rutherford Appleton Laboratory Report RAL-95-001, Oxfordshire, UK, Jan. 1995). Second derivatives of all problem functions are required when the true Hessian option is used. A Fortran implementation is briefly described. Computational results are presented for 34 smaller models coded in Fortran, where first and second derivatives are approximated by differencing, and for 89 larger GAMS models, where analytic first derivatives are available and finite differencing is used for second partials. The GAMS results are, to our knowledge, the first to show the performance of this promising class of algorithms on large sparse NLP's. For both small and large problems, both true Hessian and quasi- Newton options are quite reliable and converge rapidly. Using the true Hessian, INTOPT is as reliable as MINOS on the GAMS models, although not as reliable as CONOPT. Computation times are considerably longer than for the other 2 solvers. However, interior point methods should be considerably faster than they are here when analytic second derivatives are available, and algorithmic improvements and problem preprocessing should further narrow the gap.     

Adaptive Local Linear Quantile Regression

In this paper we propose a new method of local linear adaptive smoothing for nonparametric conditional quantile regression. Some theoretical properties of the procedure are investigated. Then we demonstrate the performance of the method on a simulated example and compare it with other methods. The simulation results demonstrate a reasonable performance of our method proposed especially in situations when the underlying image is piecewise linear or can be approximated by such images. Generally speaking, our ...     

线性互补问题的宽邻域预估校正算法

对线性互补问题提出了一种新的宽邻域预估校正算法,算法是基于经典线性规划路径跟踪算法的思想,将Maziar Salahi关于线性规划预估校正算法推广到线性互补问题中,给出了算法的具体迭代步骤并讨论了算法迭代复杂性,最后证明了算法具有多项式复杂性为O(ηlog(X~0)~Ts~0/ε)。     

Regression Adjustment for Noncrossing Bayesian Quantile Regression

A two-stage approach is proposed to overcome the problem in quantile regression, where separately fitted curves for several quantiles may cross. The standard Bayesian quantile regression model is applied in the first stage, followed by a Gaussian process regression adjustment, which monotonizes the quantile function while borrowing strength from nearby quantiles. The two-stage approach is computationally efficient, and more general than existing techniques. The method is shown to be competitive with alternative approaches via its performance in simulated examples. Supplementary materials for the article are available online.     

Quantile regression for robust bank efficiency score estimation

We discuss quantile regression techniques as a robust and easy to implement alternative for estimating Farell technical efficiency scores. The quantile regression approach estimates the production process for benchmark banks located at top conditional quantiles. Monte Carlo simulations reveal that even when generating data according to the assumptions of the stochastic frontier model (SFA), efficiency estimates obtained from quantile regressions resemble SFA-efficiency estimates. We apply the SFA and the quantile regression approach to German bank data for three banking groups, commercial banks, savings banks and cooperative banks to estimate efficiency scores based on a simple value added function and a multiple-input–multiple-output cost function. The results reveal that the efficient (benchmark) banks have production and cost elasticities which differ considerably from elasticities obtained from conditional mean functions and stochastic frontier functions.     

Pyramid Quantile Regression

We describe a Bayesian model for simultaneous linear quantile regression at several specified quantile levels. More specifically, we propose to model the conditional distributions by using random probability measures, known as quantile pyramids, introduced by Hjort and Walker. Unlike many existing approaches, this framework allows us to specify meaningful priors on the conditional distributions, while retaining the flexibility afforded by the nonparametric error distribution formulation. Simulation studies demonstrate the flexibility of the proposed approach in estimating diverse scenarios, generally outperforming other competitive methods. We also provide conditions for posterior consistency. The method is particularly promising for modeling the extremal quantiles. Applications to extreme value analysis and in higher dimensions are also explored through data examples. Supplemental material for this article is available online.     

��Ҷ˹���Ϸ�λ�ع��Gibbs�����㷨

Most regression modeling is based on traditional mean regression which results in non-robust estimation results for non-normal errors. Compared to conventional mean regression, composite quantile regression (CQR) may produce more robust parameters estimation. Based on a composite asymmetric Laplace distribution (CALD), we build a Bayesian hierarchical model for the weighted CQR (WCQR). The Gibbs sampler algorithm of Bayesian WCQR is developed to implement posterior inference. Finally, the proposed method are illustrated by some simulation studies and a real data analysis.     

�����������ĺ���������ģ�͵ĸ��Ϸ�λ������

??n this paper, we propose composite quantile regression for functional linear model with dependent data, in which the errors are from a short-range dependent and strictly stationary linear process. The functional principal component analysis is employed to approximate the slope function and the functional predictive variable respectively to construct an estimator of the slope function, and the convergence rate of the estimator is obtained under some regularity conditions. Simulation studies and a real data analysis are presented for illustration of the performance of the proposed estimator.     

Composite Quantile Regression for Functional Linear Models with Dependent Errors

n this paper, we propose composite quantile regression for functional linear model with dependent data, in which the errors are from a short-range dependent and strictly stationary linear process. The functional principal component analysis is employed to approximate the slope function and the functional predictive variable respectively to construct an estimator of the slope function, and the convergence rate of the estimator is obtained under some regularity conditions. Simulation studies and a real data analysis are presented for illustration of the performance of the proposed estimator.     

Semismooth Newton Coordinate Descent Algorithm for Elastic-Net Penalized Huber Loss Regression and Quantile Regression

We propose an algorithm, semismooth Newton coordinate descent (SNCD), for the elastic-net penalized Huber loss regression and quantile regression in high dimensional settings. Unlike existing coordinate descent type algorithms, the SNCD updates a regression coefficient and its corresponding subgradient simultaneously in each iteration. It combines the strengths of the coordinate descent and the semismooth Newton algorithm, and effectively solves the computational challenges posed by dimensionality and nonsmoothness. We establish the convergence properties of the algorithm. In addition, we present an adaptive version of the “strong rule” for screening predictors to gain extra efficiency. Through numerical experiments, we demonstrate that the proposed algorithm is very efficient and scalable to ultrahigh dimensions. We illustrate the application via a real data example. Supplementary materials for this article are available online.     

多元线性回归模型参数估计的递推算法及误差分析

讨论新增数据信息对多元线性回归模型的修正原理,给出不断加入新增数据信息的多元线性回归模型参数估计值的一种递推算法.利用影响因子的概念来刻画新增信息对预测误差的影响,并给出了算法和应用实例.     

An Iterative Coordinate Descent Algorithm for High-Dimensional Nonconvex Penalized Quantile Regression

We propose and study a new iterative coordinate descent algorithm (QICD) for solving nonconvex penalized quantile regression in high dimension. By permitting different subsets of covariates to be relevant for modeling the response variable at different quantiles, nonconvex penalized quantile regression provides a flexible approach for modeling high-dimensional data with heterogeneity. Although its theory has been investigated recently, its computation remains highly challenging when p is large due to the nonsmoothness of the quantile loss function and the nonconvexity of the penalty function. Existing coordinate descent algorithms for penalized least-squares regression cannot be directly applied. We establish the convergence property of the proposed algorithm under some regularity conditions for a general class of nonconvex penalty functions including popular choices such as SCAD (smoothly clipped absolute deviation) and MCP (minimax concave penalty). Our Monte Carlo study confirms that QICD substantially improves the computational speed in the p ? n setting. We illustrate the application by analyzing a microarray dataset.     

外部压力对学生的数学及科学成绩效应的纵向研究 ——- 非参数分位回归方法

过去, 许多研究主要集中在学生的经济背景与心理因素对其成绩的影响方面. 然而, 很少注意到外部压力对学生成绩的影响, 诸如来自家长与同学方面的压力. 本文重点放在这一有趣而且很重要的主题上, 利用非参数分位回归中的``双核'法对美国青年人进行了深入地纵向研究, 得到了几个很有趣的发现. 这些研究的方法、结果不光对学生家长有用, 而且对教育政策制定者与咨询者也有所裨益.     

基于固定效应的纵向数据分位点回归模型的参数估计及CDM和MSOM的等价性

研究了基于固定效应的纵向数据模分位点回归模型的参数估计及统计诊断问题.首先给出了参数估计的MM迭代算法,然后讨论了统计诊断中数据删除模型(CDM)和均值移模型(MSOM)的等价性问题,最后利用消炎镇痛药数据说明了方法的应用.     

A Geometric Build-Up Algorithm for Solving the Molecular Distance Geometry Problem with Sparse Distance Data

Nuclear magnetic resonance (NMR) structure modeling usually produces a sparse set of inter-atomic distances in protein. In order to calculate the three-dimensional structure of protein, current approaches need to estimate all other missing distances to build a full set of distances. However, the estimation step is costly and prone to introducing errors. In this report, we describe a geometric build-up algorithm for solving protein structure by using only a sparse set of inter-atomic distances. Such a sparse set of distances can be obtained by combining NMR data with our knowledge on certain bond lengths and bond angles. It can also include confident estimations on some missing distances. Our algorithm utilizes a simple geometric relationship between coordinates and distances. The coordinates for each atom are calculated by using the coordinates of previously determined atoms and their distances. We have implemented the algorithm and tested it on several proteins. Our results showed that our algorithm successfully determined the protein structures with sparse sets of distances. Therefore, our algorithm reduces the need of estimating the missing distances and promises a more efficient approach to NMR structure modeling.     

基于Monte Carlo模拟比较K近邻和局部线性分位数回归

局部线性分位数回归是目前比较流行的非参数分位数回归,其潜在假定待估函数线性光滑.K近邻分位数回归也是非参数分位数回归的重要组成部分,其具有不需待估函数光滑和不同分位点的回归曲线不相交等优点.通过Monte Carlo模拟,比较了两者的估计,得到当待估函数的跳跃点或突变点比较多时,K近邻分位数回归的拟合效果优于局部线性回归.其中模拟的函数是Blocks、Bumps和HeaviSine的函数,它们分别代表跳跃性、波动性、斜率突变性的函数.     

��λ���ع�Ĺ⻬������Ȼ

For linear quantile regression model, this paper proves that the test statistics, besed on smoothed empirical likelihood (SEL) method and least absolute deviation (LAD) method, both converge weakly to a noncentral Chi-square distribution under the local alternatives $H_1:beta=beta_0+a_n$, where $beta$ is the true parameter. Simulation results show that the SEL method is more efficient than the LAD method.