Adaptive quantile regression with precise risk bounds

An adaptive local smoothing method for nonparametric conditional quantile regression models is considered in this paper. Theoretical properties of the procedure are examined. The proposed method is fully adaptive in the sense that no prior information about the structure of the model is assumed. The fully adaptive feature not only allows varying bandwidths to accommodate jumps or instantaneous slope changes, but also allows the algorithm to be spatially adaptive. Under general conditions, precise risk bounds for homogeneous and heterogeneous cases of the underlying conditional quantile curves are established. An automatic selection algorithm for locally adaptive bandwidths is also given, which is applicable to higher dimensional cases. Simulation studies and data analysis confirm that the proposed methodology works well.     

Quantile regression with ℓ 1—regularization and Gaussian kernels

The quantile regression problem is considered by learning schemes based on ? ₁—regularization and Gaussian kernels. The purpose of this paper is to present concentration estimates for the algorithms. Our analysis shows that the convergence behavior of ? ₁—quantile regression with Gaussian kernels is almost the same as that of the RKHS-based learning schemes. Furthermore, the previous analysis for kernel-based quantile regression usually requires that the output sample values are uniformly bounded, which excludes the common case with Gaussian noise. Our error analysis presented in this paper can give satisfactory convergence rates even for unbounded sampling processes. Besides, numerical experiments are given which support the theoretical results.     

Consistency of kernel‐based quantile regression

Quantile regression is used in many areas of applied research and business. Examples are actuarial, financial or biometrical applications. We show that a non‐parametric generalization of quantile regression based on kernels shares with support vector machines the property of consistency to the Bayes risk. We further use this consistency to prove that the non‐parametric generalization approximates the conditional quantile function which gives the mathematical justification for kernel‐based quantile regression. Copyright © 2008 John Wiley & Sons, Ltd.     

Bayesian analysis of quantile regression for censored dynamic panel data

This paper develops a Bayesian approach to analyzing quantile regression models for censored dynamic panel data. We employ a likelihood-based approach using the asymmetric Laplace error distribution and introduce lagged observed responses into the conditional quantile function. We also deal with the initial conditions problem in dynamic panel data models by introducing correlated random effects into the model. For posterior inference, we propose a Gibbs sampling algorithm based on a location-scale mixture representation of the asymmetric Laplace distribution. It is shown that the mixture representation provides fully tractable conditional posterior densities and considerably simplifies existing estimation procedures for quantile regression models. In addition, we explain how the proposed Gibbs sampler can be utilized for the calculation of marginal likelihood and the modal estimation. Our approach is illustrated with real data on medical expenditures.     

��ָ���λ���ع�ı���ѡ��

多元非参数分位数回归常常是难于估计的, 为了降低维数同时保持非参数估计的灵活性, 人们常常用单指标的方法模拟响应变量的条件分位数. 本文主要研究单指标分位数回归的变量选择. 以最小化平均损失估计为基础, 我们通过最小化具有SCAD惩罚项的平均损失进行变量选择和参数估计. 在正则条件下, 得到了单指标分位数回归SCAD变量选择的Oracle性质, 给出了SCAD变量选择的计算方法, 并通过模拟研究说明了本文所提方法变量选择的样本性质.     

Bayesian lasso binary quantile regression

In this paper, a Bayesian hierarchical model for variable selection and estimation in the context of binary quantile regression is proposed. Existing approaches to variable selection in a binary classification context are sensitive to outliers, heteroskedasticity or other anomalies of the latent response. The method proposed in this study overcomes these problems in an attractive and straightforward way. A Laplace likelihood and Laplace priors for the regression parameters are proposed and estimated with Bayesian Markov Chain Monte Carlo. The resulting model is equivalent to the frequentist lasso procedure. A conceptional result is that by doing so, the binary regression model is moved from a Gaussian to a full Laplacian framework without sacrificing much computational efficiency. In addition, an efficient Gibbs sampler to estimate the model parameters is proposed that is superior to the Metropolis algorithm that is used in previous studies on Bayesian binary quantile regression. Both the simulation studies and the real data analysis indicate that the proposed method performs well in comparison to the other methods. Moreover, as the base model is binary quantile regression, a much more detailed insight in the effects of the covariates is provided by the approach. An implementation of the lasso procedure for binary quantile regression models is available in the R-package bayesQR.     

A New Approach to Censored Quantile Regression Estimation

Quantile regression provides an attractive tool to the analysis of censored responses, because the conditional quantile functions are often of direct interest in regression analysis, and moreover, the quantiles are often identifiable while the conditional mean functions are not. Existing methods of estimation for censored quantiles are mostly limited to singly left- or right-censored data, with some attempts made to extend the methods to doubly censored data. In this article, we propose a new and unified approach, based on a variation of the data augmentation algorithm, to censored quantile regression estimation. The proposed method adapts easily to different forms of censoring including doubly censored and interval censored data, and somewhat surprisingly, the resulting estimates improve on the performance of the best known estimators with singly censored data. Supplementary material for this article is available online.     

Quantile regression for robust bank efficiency score estimation

We discuss quantile regression techniques as a robust and easy to implement alternative for estimating Farell technical efficiency scores. The quantile regression approach estimates the production process for benchmark banks located at top conditional quantiles. Monte Carlo simulations reveal that even when generating data according to the assumptions of the stochastic frontier model (SFA), efficiency estimates obtained from quantile regressions resemble SFA-efficiency estimates. We apply the SFA and the quantile regression approach to German bank data for three banking groups, commercial banks, savings banks and cooperative banks to estimate efficiency scores based on a simple value added function and a multiple-input–multiple-output cost function. The results reveal that the efficient (benchmark) banks have production and cost elasticities which differ considerably from elasticities obtained from conditional mean functions and stochastic frontier functions.     

Regression Adjustment for Noncrossing Bayesian Quantile Regression

A two-stage approach is proposed to overcome the problem in quantile regression, where separately fitted curves for several quantiles may cross. The standard Bayesian quantile regression model is applied in the first stage, followed by a Gaussian process regression adjustment, which monotonizes the quantile function while borrowing strength from nearby quantiles. The two-stage approach is computationally efficient, and more general than existing techniques. The method is shown to be competitive with alternative approaches via its performance in simulated examples. Supplementary materials for the article are available online.     

Modal Regression Based on Nonparametric Quantile Estimator

Modal regression based on nonparametric quantile estimator is given. Unlike the traditional mean and median regression, modal regression uses mode but not mean or median to represent the center of a conditional distribution, which helps the model to be more robust for outliers, asymmetric or heavy-taileddistribution. Most of solutions for modal regression are based on kernel estimation of density. This paper studies a new solution for modal regression by means of nonparametric quantile estimator. This method builds on the fact that the distribution function is the inverse of the quantile function, then the flexibility of nonparametric quantile estimator is utilized to improve the estimation of modal function. The simulations and application show that the new model outperforms the modal regression model via linear quantile function estimation.     

Weighted quantile regression for longitudinal data using empirical likelihood

This paper proposes a new weighted quantile regression model for longitudinal data with weights chosen by empirical likelihood (EL). This approach efficiently incorporates the information from the conditional quantile restrictions to account for within-subject correlations. The resulted estimate is computationally simple and has good performance under modest or high within-subject correlation. The efficiency gain is quantified theoretically and illustrated via simulation and a real data application.     

Conditional Quantile Estimation with Truncated, Censored and Dependent Data

This paper deals with the conditional quantile estimation based on left-truncated and right-censored data.Assuming that the observations with multivariate covariates form a stationary α-mixing sequence,the authors derive the strong convergence with rate,strong representation as well as asymptotic normality of the conditional quantile estimator.Also,a Berry-Esseen-type bound for the estimator is established.In addition,the finite sample behavior of the estimator is investigated via simulations.     

Estimation of additive quantile regression

We consider the nonparametric estimation problem of conditional regression quantiles with high-dimensional covariates. For the additive quantile regression model, we propose a new procedure such that the estimated marginal effects of additive conditional quantile curves do not cross. The method is based on a combination of the marginal integration technique and non-increasing rearrangements, which were recently introduced in the context of estimating a monotone regression function. Asymptotic normality of the estimates is established with a one-dimensional rate of convergence and the finite sample properties are studied by means of a simulation study and a data example.     

Unified Noncrossing Multiple Quantile Regressions Tree

In this article, we consider the estimation problem of a tree model for multiple conditional quantile functions of the response. Using the generalized, unbiased interaction detection and estimation algorithm, the quantile regression tree (QRT) method has been developed to construct a tree model for an individual quantile function. However, QRT produces different tree models across quantile levels because it estimates several QRT models separately. Furthermore, the estimated quantile functions from QRT often cross each other and consequently violate the basic properties of quantiles. This undesirable phenomenon reduces prediction accuracy and makes it difficult to interpret the resulting tree models. To overcome such limitations, we propose the unified noncrossing multiple quantile regressions tree (UNQRT) method, which constructs a common tree structure across all interesting quantile levels for better data visualization and model interpretation. Furthermore, the UNQRT estimates noncrossing multiple quantile functions simultaneously by enforcing noncrossing constraints, resulting in the improvement of prediction accuracy. The numerical results are presented to demonstrate the competitive performance of the proposed UNQRT over QRT. Supplementary materials for this article are available online.     

无界意义下的在线变化分位数回归算法

本文研究了基于核方法下的在线变化损失函数的回归算法. 利用迭代和比较原则, 得到了算法的收敛速度, 并将该结果推广到了更一般的输出空间.     

无界意义下的在线变化分位数回归算法

本文研究了基于核方法下的在线变化损失函数的回归算法.利用迭代和比较原则,得到了算法的收敛速度,并将该结果推广到了更一般的输出空间.     

Cover-based bounds on the numerical rank of Gaussian kernels

A popular approach for analyzing high-dimensional datasets is to perform dimensionality reduction by applying non-parametric affinity kernels. Usually, it is assumed that the represented affinities are related to an underlying low-dimensional manifold from which the data is sampled. This approach works under the assumption that, due to the low-dimensionality of the underlying manifold, the kernel has a low numerical rank. Essentially, this means that the kernel can be represented by a small set of numerically-significant eigenvalues and their corresponding eigenvectors.We present an upper bound for the numerical rank of Gaussian convolution operators, which are commonly used as kernels by spectral manifold-learning methods. The achieved bound is based on the underlying geometry that is provided by the manifold from which the dataset is assumed to be sampled. The bound can be used to determine the number of significant eigenvalues/eigenvectors that are needed for spectral analysis purposes. Furthermore, the results in this paper provide a relation between the underlying geometry of the manifold (or dataset) and the numerical rank of its Gaussian affinities.The term cover-based bound is used because the computations of this bound are done by using a finite set of small constant-volume boxes that cover the underlying manifold (or the dataset). We present bounds for finite Gaussian kernel matrices as well as for the continuous Gaussian convolution operator. We explore and demonstrate the relations between the bounds that are achieved for finite and continuous cases. The cover-oriented methodology is also used to provide a relation between the geodesic length of a curve and the numerical rank of Gaussian kernel of datasets that are sampled from it.     

Nonparametric quantile regression with heavy-tailed and strongly dependent errors

We consider nonparametric estimation of the conditional qth quantile for stationary time series. We deal with stationary time series with strong time dependence and heavy tails under the setting of random design. We estimate the conditional qth quantile by local linear regression and investigate the asymptotic properties. It is shown that the asymptotic properties are affected by both the time dependence and the tail index of the errors. The results of a small simulation study are also given.     

Pyramid Quantile Regression

We describe a Bayesian model for simultaneous linear quantile regression at several specified quantile levels. More specifically, we propose to model the conditional distributions by using random probability measures, known as quantile pyramids, introduced by Hjort and Walker. Unlike many existing approaches, this framework allows us to specify meaningful priors on the conditional distributions, while retaining the flexibility afforded by the nonparametric error distribution formulation. Simulation studies demonstrate the flexibility of the proposed approach in estimating diverse scenarios, generally outperforming other competitive methods. We also provide conditions for posterior consistency. The method is particularly promising for modeling the extremal quantiles. Applications to extreme value analysis and in higher dimensions are also explored through data examples. Supplemental material for this article is available online.     

Nonlinear approximation using Gaussian kernels

It is well known that nonlinear approximation has an advantage over linear schemes in the sense that it provides comparable approximation rates to those of the linear schemes, but to a larger class of approximands. This was established for spline approximations and for wavelet approximations, and more recently by DeVore and Ron (in press) [2] for homogeneous radial basis function (surface spline) approximations. However, no such results are known for the Gaussian function, the preferred kernel in machine learning and several engineering problems. We introduce and analyze in this paper a new algorithm for approximating functions using translates of Gaussian functions with varying tension parameters. At heart it employs the strategy for nonlinear approximation of DeVore-Ron, but it selects kernels by a method that is not straightforward. The crux of the difficulty lies in the necessity to vary the tension parameter in the Gaussian function spatially according to local information about the approximand: error analysis of Gaussian approximation schemes with varying tension are, by and large, an elusive target for approximators. We show that our algorithm is suitably optimal in the sense that it provides approximation rates similar to other established nonlinear methodologies like spline and wavelet approximations. As expected and desired, the approximation rates can be as high as needed and are essentially saturated only by the smoothness of the approximand.