Nonparametric regression under dependent errors with infinite variance

We consider local least absolute deviation (LLAD) estimation for trend functions of time series with heavy tails which are characterised via a symmetric stable law distribution. The setting includes both causal stable ARMA model and fractional stable ARIMA model as special cases. The asymptotic limit of the estimator is established under the assumption that the process has either short or long memory autocorrelation. For a short memory process, the estimator admits the same convergence rate as if the process has the finite variance. The optimal rate of convergencen ^−2/5 is obtainable by using appropriate bandwidths. This is distinctly different from local least squares estimation, of which the convergence is slowed down due to the existence of heavy tails. On the other hand, the rate of convergence of the LLAD estimator for a long memory process is always slower thann ^−2/5 and the limit is no longer normal.     

Nonparametric regression with martingale increment errors

We consider the problem of adaptive estimation of the regression function in a framework where we replace ergodicity assumptions (such as independence or mixing) by another structural assumption on the model. Namely, we propose adaptive upper bounds for kernel estimators with data-driven bandwidth (Lepski’s selection rule) in a regression model where the noise is an increment of martingale. It includes, as very particular cases, the usual i.i.d. regression and auto-regressive models. The cornerstone tool for this study is a new result for self-normalized martingales, called “stability”, which is of independent interest. In a first part, we only use the martingale increment structure of the noise. We give an adaptive upper bound using a random rate, that involves the occupation time near the estimation point. Thanks to this approach, the theoretical study of the statistical procedure is disconnected from usual ergodicity properties like mixing. Then, in a second part, we make a link with the usual minimax theory of deterministic rates. Under a β-mixing assumption on the covariates process, we prove that the random rate considered in the first part is equivalent, with large probability, to a deterministic rate which is the usual minimax adaptive one.     

Composite quantile regression for single-index models with asymmetric errors

For the single-index model, a composite quantile regression technique is proposed in this paper to construct robust and efficient estimation. Theoretical analysis reveals that the proposed estimate of the single-index vector is highly efficient relative to its corresponding least squares estimate. For the single-index vector, the proposed method is always valid across a wide spectrum of error distributions; even in the worst case scenario, the asymptotic relative efficiency has a lower bound 86.4 %. Meanwhile, we employ weighted local composite quantile regression to obtain a consistent and robust estimate for the nonparametric component in the single-index model, which is adapted to both symmetric and asymmetric distributions. Numerical study and a real data analysis can further illustrate our theoretical findings.     

On Monte Carlo methods for Bayesian multivariate regression models with heavy-tailed errors

We consider Bayesian analysis of data from multivariate linear regression models whose errors have a distribution that is a scale mixture of normals. Such models are used to analyze data on financial returns, which are notoriously heavy-tailed. Let π denote the intractable posterior density that results when this regression model is combined with the standard non-informative prior on the unknown regression coefficients and scale matrix of the errors. Roughly speaking, the posterior is proper if and only if n≥d+k, where n is the sample size, d is the dimension of the response, and k is number of covariates. We provide a method of making exact draws from π in the special case where n=d+k, and we study Markov chain Monte Carlo (MCMC) algorithms that can be used to explore π when n>d+k. In particular, we show how the Haar PX-DA technology studied in Hobert and Marchev (2008) [11] can be used to improve upon Liu’s (1996) [7] data augmentation (DA) algorithm. Indeed, the new algorithm that we introduce is theoretically superior to the DA algorithm, yet equivalent to DA in terms of computational complexity. Moreover, we analyze the convergence rates of these MCMC algorithms in the important special case where the regression errors have a Student’s t distribution. We prove that, under conditions on n, d, k, and the degrees of freedom of the t distribution, both algorithms converge at a geometric rate. These convergence rate results are important from a practical standpoint because geometric ergodicity guarantees the existence of central limit theorems which are essential for the calculation of valid asymptotic standard errors for MCMC based estimates.     

Bayesian joint quantile regression for mixed effects models with censoring and errors in covariates

In this paper, we discuss Bayesian joint quantile regression of mixed effects models with censored responses and errors in covariates simultaneously using Markov Chain Monte Carlo method. Under the assumption of asymmetric Laplace error distribution, we establish a Bayesian hierarchical model and derive the posterior distributions of all unknown parameters based on Gibbs sampling algorithm. Three cases including multivariate normal distribution and other two heavy-tailed distributions are considered for fitting random effects of the mixed effects models. Finally, some Monte Carlo simulations are performed and the proposed procedure is illustrated by analyzing a group of AIDS clinical data set.     

A new extreme quantile estimator for heavy-tailed distributions

The classical estimation method for extreme quantiles of heavy-tailed distributions was presented by Weissman (J. Amer. Statist. Assoc. 73 (1978) 812–815) and makes use of the Hill estimator (Ann. Statist. 3 (1975) 1163–1174) for the positive extreme value index. This index estimator can be interpreted as an estimator of the slope in the Pareto quantile plot in case one considers regression lines passing through a fixed anchor point. In this Note we propose a new extreme quantile estimator based on an unconstrained least squares estimator of the index, introduced by Kratz and Resnick (Comm. Statist. Stochastic Models 12 (1996) 699–724) and Schultze and Steinebach (Statist. Decisions 14 (1996) 353–372) and we study its asymptotic behavior. To cite this article: A. Fils, A. Guillou, C. R. Acad. Sci. Paris, Ser. I 338 (2004).     

Nonparametric regression estimation for dependent functional data: asymptotic normality

We consider the estimation of a regression functional where the explanatory variables take values in some abstract function space. The principal aim of the paper is to establish the asymptotic normality of such estimates for dependent functional data.     

On linear models with long memory and heavy-tailed errors

We consider the robust estimation of regression parameters in linear models with long memory and heavy-tailed errors. Asymptotic Bahadur-type representations of robust estimates are developed and their limiting distributions are obtained. It is shown that the limiting distributions are very different from those obtained under short memory. A simulation study is carried out to compare the performance of various asymptotic representations.     

Single-index quantile regression

Nonparametric quantile regression with multivariate covariates is a difficult estimation problem due to the “curse of dimensionality”. To reduce the dimensionality while still retaining the flexibility of a nonparametric model, we propose modeling the conditional quantile by a single-index function , where a univariate link function g₀(⋅) is applied to a linear combination of covariates , often called the single-index. We introduce a practical algorithm where the unknown link function g₀(⋅) is estimated by local linear quantile regression and the parametric index is estimated through linear quantile regression. Large sample properties of estimators are studied, which facilitate further inference. Both the modeling and estimation approaches are demonstrated by simulation studies and real data applications.     

Semiparametric quantile regression with random censoring

This paper considers estimation and inference in semiparametric quantile regression models when the response variable is subject to random censoring. The paper considers both the cases of independent and dependent censoring and proposes three iterative estimators based on inverse probability weighting, where the weights are estimated from the censoring distribution using the Kaplan–Meier, a fully parametric and the conditional Kaplan–Meier estimators. The paper proposes a computationally simple resampling technique that can be used to approximate the finite sample distribution of the parametric estimator. The paper also considers inference for both the parametric and nonparametric components of the quantile regression model. Monte Carlo simulations show that the proposed estimators and test statistics have good finite sample properties. Finally, the paper contains a real data application, which illustrates the usefulness of the proposed methods.

     

Nonparametric regression with interval-censored data

In many medical studies,the prevalence of interval censored data is increasing due to periodic monitoring of the progression status of a disease.In nonparametric regression model,when the response variable is subjected to interval-censoring,the regression function could not be estimated by traditional methods directly.With the censored data,we construct a new response variable which has the same conditional expectation as the original one.Based on the new variable,we get a nearest neighbor estimator of the regression function.It is established that the estimator has strong consistency and asymptotic normality.The relevant simulation reports are given.     

Nonparametric quantile estimation using importance sampling

Nonparametric estimation of a quantile of a random variable m(X) is considered, where \(m: \mathbb {R}^d\rightarrow \mathbb {R}\) is a function which is costly to compute and X is a \(\mathbb {R}^d\)-valued random variable with a given density. An importance sampling quantile estimate of m(X), which is based on a suitable estimate \(m_n\) of m, is defined, and it is shown that this estimate achieves a rate of convergence of order \(\log ^{1.5}(n)/n\). The finite sample size behavior of the estimate is illustrated by simulated data.     

Asymptotics for partly linear regression with dependent samples and ARCH errors: consistency with rates

Partly linear regression model is useful in practice, but little is investigated in the literature to adapt it to the real data which are dependent and conditionally heteroscedastic. In this paper, the estimators of the regression components are constructed via local polynomial fitting and the large Sample properties are explored. Under certain mild regularities, the conditions are obtained to ensure that the estimators of the nonparametric component and its derivatives are consistent up to the convergence rates which are optimal in the i. i. d. case, and the estimator of the parametric component is root-n consistent with the same rate as for parametric model. The technique adopted in the proof differs from that used and corrects the errors in the reference by Hamilton and Truong under i. i. d. samples.     

Adaptive quantile regression with precise risk bounds

An adaptive local smoothing method for nonparametric conditional quantile regression models is considered in this paper. Theoretical properties of the procedure are examined. The proposed method is fully adaptive in the sense that no prior information about the structure of the model is assumed. The fully adaptive feature not only allows varying bandwidths to accommodate jumps or instantaneous slope changes, but also allows the algorithm to be spatially adaptive. Under general conditions, precise risk bounds for homogeneous and heterogeneous cases of the underlying conditional quantile curves are established. An automatic selection algorithm for locally adaptive bandwidths is also given, which is applicable to higher dimensional cases. Simulation studies and data analysis confirm that the proposed methodology works well.     

Risk factor extraction with quantile regression method

Firm characteristics based risk factors constitute a large part of the asset pricing literature. These characteristic based factors are constructed using the extreme quantiles of the sorted portfolios based on the firm characteristic in question. Yet to date, there is no consensus on a systematic approach to determine the optimal quantile used for extracting firm characteristic based risk factors. In addition, it is a stylised fact that asset prices exhibit heteroscedastic behavior, and counting on the extreme portfolios to extract the characteristic factors can produce unexpected result. In this study, we use quantile regressions to determine the optimal quantiles used in portfolios sorts to extract characteristic based risk factors used in asset pricing. Quantile regressions are well-suited to identify the quantiles needed to extract firm characteristic based factors, especially when the firm characteristic based factors and stock returns relationship is non-linear. More over, quantile regressions presents the quantile-by-quantile risk-return coefficients, thereby verifying the behavior of the extreme quantiles used in the factor construction. By examining the relationship between common characteristic based factors and stock returns in 23 developed countries, we observed that the optimal quantiles used to construct the common factors may differ between factors, but is similar across the North American, Asia-Pacific and Europe regions.

     

Support vector quantile regression with varying coefficients

Quantile regression has received a great deal of attention as an important tool for modeling statistical quantities of interest other than the conditional mean. Varying coefficient models are widely used to explore dynamic patterns among popular models available to avoid the curse of dimensionality. We propose a support vector quantile regression model with varying coefficients and its two estimation methods. One uses the quadratic programming, and the other uses the iteratively reweighted least squares procedure. The proposed method can be applied easily and effectively to estimating the nonlinear regression quantiles depending on the high-dimensional vector of smoothing variables. We also present the model selection method that employs generalized cross validation and generalized approximate cross validation techniques for choosing the hyperparameters, which affect the performance of the proposed model. Numerical studies are conducted to illustrate the performance of the proposed model.     

Averaged extreme regression quantile

Various events in the nature, economics and in other areas force us to combine the study of extremes with regression and other methods. A useful tool for reducing the role of nuisance regression, while we are interested in the shape or tails of the basic distribution, is provided by the averaged regression quantile and namely by the average extreme regression quantile. Both are weighted means of regression quantile components, with weights depending on the regressors. Our primary interest is the averaged extreme regression quantile (AERQ), its structure, qualities and its applications, e.g. in investigation of a conditional loss given a value exogenous economic and market variables. AERQ has several interesting equivalent forms: While it is originally defined as an optimal solution of a specific linear programming problem, hence is a weighted mean of responses corresponding to the optimal base of the pertaining linear program, we give another equivalent form as a maximum residual of responses from a specific R-estimator of the slope components of regression parameter. The latter form shows that while AERQ equals to the maximum of some residuals of the responses, it has minimal possible perturbation by the regressors. Notice that these finite-sample results are true even for non-identically distributed model errors, e.g. under heteroscedasticity. Moreover, the representations are formally true even when the errors are dependent - this all provokes a question of the right interpretation and of other possible applications.     

Nonparametric quantile frontier estimation under shape restriction

This paper proposes a shape-restricted nonparametric quantile regression to estimate the τ-frontier, which acts as a benchmark for whether a decision making unit achieves top τ efficiency. This method adopts a two-step strategy: first, identifying fitted values that minimize an asymmetric absolute loss under the nondecreasing and concave shape restriction; second, constructing a nondecreasing and concave estimator that links these fitted values. This method makes no assumption on the error distribution and the functional form. Experimental results on some artificial data sets clearly demonstrate its superiority over the classical linear quantile regression. We also discuss how to enforce constraints to avoid quantile crossings between multiple estimated frontiers with different values of τ. Finally this paper shows that this method can be applied to estimate the production function when one has some prior knowledge about the error term.