Composite support vector quantile regression estimation

In this paper we propose a new nonparametric regression method called composite support vector quantile regression (CSVQR) that combines the formulations of support vector regression and composite quantile regression. First the CSVQR using the quadratic programming (QP) is proposed and then the CSVQR utilizing the iteratively reweighted least squares (IRWLS) procedure is proposed to overcome weakness of the QP based method in terms of computation time. The IRWLS procedure based method enables us to derive a generalized cross validation (GCV) function that is easier and faster than the conventional cross validation function. The GCV function facilitates choosing the hyperparameters that affect the performance of the CSVQR and saving computation time. Numerical experiment results are presented to illustrate the performance of the proposed method     

Marginal quantile regression for varying coefficient models with longitudinal data

In this paper, we investigate the quantile varying coefficient model for longitudinal data, where the unknown nonparametric functions are approximated by polynomial splines and the estimators are obtained by minimizing the quadratic inference function. The theoretical properties of the resulting estimators are established, and they achieve the optimal convergence rate for the nonparametric functions. Since the objective function is non-smooth, an estimation procedure is proposed that uses induced smoothing and we prove that the smoothed estimator is asymptotically equivalent to the original estimator. Moreover, we propose a variable selection procedure based on the regularization method, which can simultaneously estimate and select important nonparametric components and has the asymptotic oracle property. Extensive simulations and a real data analysis show the usefulness of the proposed method.

     

Estimating value at risk with semiparametric support vector quantile regression

Value at Risk (VaR) has been used as an important tool to measure the market risk under normal market. Usually the VaR of log returns is calculated by assuming a normal distribution. However, log returns are frequently found not normally distributed. This paper proposes the estimation approach of VaR using semiparametric support vector quantile regression (SSVQR) models which are functions of the one-step-ahead volatility forecast and the length of the holding period, and can be used regardless of the distribution. We find that the proposed models perform better overall than the variance-covariance and linear quantile regression approaches for return data on S&P 500, NIKEI 225 and KOSPI 200 indices.     

Support vector machines regression with unbounded sampling

Uniform boundedness of output variables is a standard assumption in most theoretical analysis of regression algorithms. This standard assumption has recently been weaken to a moment hypothesis in least square regression (LSR) setting. Although there has been a large literature on error analysis for LSR under the moment hypothesis, very little is known about the statistical properties of support vector machines regression with unbounded sampling. In this paper, we fill the gap in the literature. Without any restriction on the boundedness of the output sampling, we establish an ad hoc convergence analysis for support vector machines regression under very mild conditions.     

Support vector machines regression with l1-regularizer

The classical support vector machines regression (SVMR) is known as a regularized learning algorithm in reproducing kernel Hilbert spaces (RKHS) with a $\epsilon$-insensitive loss function and an RKHS norm regularizer. In this paper, we study a new SVMR algorithm where the regularization term is proportional to $l^1$-norm of the coefficients in the kernel ensembles. We provide an error analysis of this algorithm, an explicit learning rate is then derived under some assumptions.     

Single-index quantile regression

Nonparametric quantile regression with multivariate covariates is a difficult estimation problem due to the “curse of dimensionality”. To reduce the dimensionality while still retaining the flexibility of a nonparametric model, we propose modeling the conditional quantile by a single-index function , where a univariate link function g₀(⋅) is applied to a linear combination of covariates , often called the single-index. We introduce a practical algorithm where the unknown link function g₀(⋅) is estimated by local linear quantile regression and the parametric index is estimated through linear quantile regression. Large sample properties of estimators are studied, which facilitate further inference. Both the modeling and estimation approaches are demonstrated by simulation studies and real data applications.     

Support vector regression for warranty claim forecasting

Forecasting the number of warranty claims is vitally important for manufacturers/warranty providers in preparing fiscal plans. In existing literature, a number of techniques such as log-linear Poisson models, Kalman filter, time series models, and artificial neural network models have been developed. Nevertheless, one might find two weaknesses existing in these approaches: (1) they do not consider the fact that warranty claims reported in the recent months might be more important in forecasting future warranty claims than those reported in the earlier months, and (2) they are developed based on repair rates (i.e., the total number of claims divided by the total number of products in service), which can cause information loss through such an arithmetic-mean operation.To overcome the above two weaknesses, this paper introduces two different approaches to forecasting warranty claims: the first is a weighted support vector regression (SVR) model and the second is a weighted SVR-based time series model. These two approaches can be applied to two scenarios: when only claim rate data are available and when original claim data are available. Two case studies are conducted to validate the two modelling approaches. On the basis of model evaluation over six months ahead forecasting, the results show that the proposed models exhibit superior performance compared to that of multilayer perceptrons, radial basis function networks and ordinary support vector regression models.     

Semiparametric quantile regression with random censoring

This paper considers estimation and inference in semiparametric quantile regression models when the response variable is subject to random censoring. The paper considers both the cases of independent and dependent censoring and proposes three iterative estimators based on inverse probability weighting, where the weights are estimated from the censoring distribution using the Kaplan–Meier, a fully parametric and the conditional Kaplan–Meier estimators. The paper proposes a computationally simple resampling technique that can be used to approximate the finite sample distribution of the parametric estimator. The paper also considers inference for both the parametric and nonparametric components of the quantile regression model. Monte Carlo simulations show that the proposed estimators and test statistics have good finite sample properties. Finally, the paper contains a real data application, which illustrates the usefulness of the proposed methods.

     

Support vector regression for loss given default modelling

Loss given default modelling has become crucially important for banks due to the requirement that they comply with the Basel Accords and to their internal computations of economic capital. In this paper, support vector regression (SVR) techniques are applied to predict loss given default of corporate bonds, where improvements are proposed to increase prediction accuracy by modifying the SVR algorithm to account for heterogeneity of bond seniorities. We compare the predictions from SVR techniques with thirteen other algorithms. Our paper has three important results. First, at an aggregated level, the proposed improved versions of support vector regression techniques outperform other methods significantly. Second, at a segmented level, by bond seniority, least square support vector regression demonstrates significantly better predictive abilities compared with the other statistical models. Third, standard transformations of loss given default do not improve prediction accuracy. Overall our empirical results show that support vector regression techniques are a promising technique for banks to use to predict loss given default.     

Risk factor extraction with quantile regression method

Firm characteristics based risk factors constitute a large part of the asset pricing literature. These characteristic based factors are constructed using the extreme quantiles of the sorted portfolios based on the firm characteristic in question. Yet to date, there is no consensus on a systematic approach to determine the optimal quantile used for extracting firm characteristic based risk factors. In addition, it is a stylised fact that asset prices exhibit heteroscedastic behavior, and counting on the extreme portfolios to extract the characteristic factors can produce unexpected result. In this study, we use quantile regressions to determine the optimal quantiles used in portfolios sorts to extract characteristic based risk factors used in asset pricing. Quantile regressions are well-suited to identify the quantiles needed to extract firm characteristic based factors, especially when the firm characteristic based factors and stock returns relationship is non-linear. More over, quantile regressions presents the quantile-by-quantile risk-return coefficients, thereby verifying the behavior of the extreme quantiles used in the factor construction. By examining the relationship between common characteristic based factors and stock returns in 23 developed countries, we observed that the optimal quantiles used to construct the common factors may differ between factors, but is similar across the North American, Asia-Pacific and Europe regions.

     

Adaptive quantile regression with precise risk bounds

An adaptive local smoothing method for nonparametric conditional quantile regression models is considered in this paper. Theoretical properties of the procedure are examined. The proposed method is fully adaptive in the sense that no prior information about the structure of the model is assumed. The fully adaptive feature not only allows varying bandwidths to accommodate jumps or instantaneous slope changes, but also allows the algorithm to be spatially adaptive. Under general conditions, precise risk bounds for homogeneous and heterogeneous cases of the underlying conditional quantile curves are established. An automatic selection algorithm for locally adaptive bandwidths is also given, which is applicable to higher dimensional cases. Simulation studies and data analysis confirm that the proposed methodology works well.     

Averaged extreme regression quantile

Various events in the nature, economics and in other areas force us to combine the study of extremes with regression and other methods. A useful tool for reducing the role of nuisance regression, while we are interested in the shape or tails of the basic distribution, is provided by the averaged regression quantile and namely by the average extreme regression quantile. Both are weighted means of regression quantile components, with weights depending on the regressors. Our primary interest is the averaged extreme regression quantile (AERQ), its structure, qualities and its applications, e.g. in investigation of a conditional loss given a value exogenous economic and market variables. AERQ has several interesting equivalent forms: While it is originally defined as an optimal solution of a specific linear programming problem, hence is a weighted mean of responses corresponding to the optimal base of the pertaining linear program, we give another equivalent form as a maximum residual of responses from a specific R-estimator of the slope components of regression parameter. The latter form shows that while AERQ equals to the maximum of some residuals of the responses, it has minimal possible perturbation by the regressors. Notice that these finite-sample results are true even for non-identically distributed model errors, e.g. under heteroscedasticity. Moreover, the representations are formally true even when the errors are dependent - this all provokes a question of the right interpretation and of other possible applications.     

Bayesian lasso binary quantile regression

In this paper, a Bayesian hierarchical model for variable selection and estimation in the context of binary quantile regression is proposed. Existing approaches to variable selection in a binary classification context are sensitive to outliers, heteroskedasticity or other anomalies of the latent response. The method proposed in this study overcomes these problems in an attractive and straightforward way. A Laplace likelihood and Laplace priors for the regression parameters are proposed and estimated with Bayesian Markov Chain Monte Carlo. The resulting model is equivalent to the frequentist lasso procedure. A conceptional result is that by doing so, the binary regression model is moved from a Gaussian to a full Laplacian framework without sacrificing much computational efficiency. In addition, an efficient Gibbs sampler to estimate the model parameters is proposed that is superior to the Metropolis algorithm that is used in previous studies on Bayesian binary quantile regression. Both the simulation studies and the real data analysis indicate that the proposed method performs well in comparison to the other methods. Moreover, as the base model is binary quantile regression, a much more detailed insight in the effects of the covariates is provided by the approach. An implementation of the lasso procedure for binary quantile regression models is available in the R-package bayesQR.     

Composite hierachical linear quantile regression

Multilevel （hierarchical） modeling is a generalization of linear and generalized linear modeling in which regression coefficients are modeled through a model, whose parameters are also estimated from data. Multilevel model fails to fit well typically by the use of the EM algorithm once one of level error variance （like Cauchy distribution） tends to infinity. This paper proposes a composite multilevel to combine the nested structure of multilevel data and the robustness of the composite quantile regression, which greatly improves the efficiency and precision of the estimation. The new approach, which is based on the Gauss-Seidel iteration and takes a full advantage of the composite quantile regression and multilevel models, still works well when the error variance tends to infinity, We show that even the error distribution is normal, the MSE of the estimation of composite multilevel quantile regression models nearly equals to mean regression. When the error distribution is not normal, our method still enjoys great advantages in terms of estimation efficiency.     

Using quantile regression for rate-making

Regression models are popular tools for rate-making in the framework of heterogeneous insurance portfolios; however, the traditional regression methods have some disadvantages particularly their sensitivity to the assumptions which significantly restrict the area of their applications. This paper is devoted to an alternative approach–quantile regression. It is free of some disadvantages of the traditional models. The quality of estimators for the approach described is approximately the same as or sometimes better than that for the traditional regression methods. Moreover, the quantile regression is consistent with the idea of using the distribution quantile for rate-making. This paper provides detailed comparisons between the approaches and it gives the practical example of using the new methodology.     

Adaptive local linear quantile regression

In this paper we propose a new method of local linear adaptive smoothing for nonparametric conditional quantile regression. Some theoretical properties of the procedure are investigated. Then we demonstrate the performance of the method on a simulated example and compare it with other methods. The simulation results demonstrate a reasonable performance of our method proposed especially in situations when the underlying image is piecewise linear or can be approximated by such images. Generally speaking, our method outperforms most other existing methods in the sense of the mean square estimation (MSE) and mean absolute estimation (MAE) criteria. The procedure is very stable with respect to increasing noise level and the algorithm can be easily applied to higher dimensional situations.     

Estimation of additive quantile regression

We consider the nonparametric estimation problem of conditional regression quantiles with high-dimensional covariates. For the additive quantile regression model, we propose a new procedure such that the estimated marginal effects of additive conditional quantile curves do not cross. The method is based on a combination of the marginal integration technique and non-increasing rearrangements, which were recently introduced in the context of estimating a monotone regression function. Asymptotic normality of the estimates is established with a one-dimensional rate of convergence and the finite sample properties are studied by means of a simulation study and a data example.     

On directional multiple-output quantile regression

This paper sheds some new light on projection quantiles. Contrary to the sophisticated set analysis used in Kong and Mizera (2008) [13], we adopt a more parametric approach and study the subgradient conditions associated with these quantiles. In this setup, we introduce Lagrange multipliers which can be interpreted in various interesting ways, in particular in a portfolio optimization context. The corresponding projection quantile regions were already shown to coincide with the halfspace depth ones in Kong and Mizera (2008) [13], but we provide here an alternative proof (completely based on projection quantiles) that has the advantage of leading to an exact computation of halfspace depth regions from projection quantiles. Above all, we systematically consider the regression case, which was barely touched in Kong and Mizera (2008) [13]. We show in particular that the regression quantile regions introduced in Hallin, Paindaveine, and Šiman (2010) [6] and [7] can also be obtained from projection (regression) quantiles, which may lead to a faster computation of those regions in some particular cases.     

Partial functional linear quantile regression

This paper studies estimation in partial functional linear quantile regression in which the dependent variable is related to both a vector of finite length and a function-valued random variable as predictor variables. The slope function is estimated by the functional principal component basis. The asymptotic distribution of the estimator of the vector of slope parameters is derived and the global convergence rate of the quantile estimator of unknown slope function is established under suitable norm. It is showed that this rate is optimal in a minimax sense under some smoothness assumptions on the covariance kernel of the covariate and the slope function. The convergence rate of the mean squared prediction error for the proposed estimators is also be established. Finite sample properties of our procedures are studied through Monte Carlo simulations. A real data example about Berkeley growth data is used to illustrate our proposed methodology.     

Nonparametric quantile regression with heavy-tailed and strongly dependent errors

We consider nonparametric estimation of the conditional qth quantile for stationary time series. We deal with stationary time series with strong time dependence and heavy tails under the setting of random design. We estimate the conditional qth quantile by local linear regression and investigate the asymptotic properties. It is shown that the asymptotic properties are affected by both the time dependence and the tail index of the errors. The results of a small simulation study are also given.