A review on consistency and robustness properties of support vector machines for heavy-tailed distributions

Support vector machines (SVMs) belong to the class of modern statistical machine learning techniques and can be described as M-estimators with a Hilbert norm regularization term for functions. SVMs are consistent and robust for classification and regression purposes if based on a Lipschitz continuous loss and a bounded continuous kernel with a dense reproducing kernel Hilbert space. For regression, one of the conditions used is that the output variable Y has a finite first absolute moment. This assumption, however, excludes heavy-tailed distributions. Recently, the applicability of SVMs was enlarged to these distributions by considering shifted loss functions. In this review paper, we briefly describe the approach of SVMs based on shifted loss functions and list some properties of such SVMs. Then, we prove that SVMs based on a bounded continuous kernel and on a convex and Lipschitz continuous, but not necessarily differentiable, shifted loss function have a bounded Bouligand influence function for all distributions, even for heavy-tailed distributions including extreme value distributions and Cauchy distributions. SVMs are thus robust in this sense. Our result covers the important loss functions ${\epsilon}$ -insensitive for regression and pinball for quantile regression, which were not covered by earlier results on the influence function. We demonstrate the usefulness of SVMs even for heavy-tailed distributions by applying SVMs to a simulated data set with Cauchy errors and to a data set of large fire insurance claims of Copenhagen Re.     

Nonparametric Estimation of Extreme Conditional Quantiles with Functional Covariate

Estimation of the extreme conditional quantiles with functional covariate is an important problem in quantile regression. The existing methods, however, are only applicable for heavy-tailed distributions with a positive conditional tail index. In this paper, we propose a new framework for estimating the extreme conditional quantiles with functional covariate that combines the nonparametric modeling techniques and extreme value theory systematically. Our proposed method is widely applicable, no matter whether the conditional distribution of a response variable Y given a vector of functional covariates X is short, light or heavy-tailed. It thus enriches the existing literature.     

多指标面板数据的聚类分析及其应用

多指标面板数据的多元统计分析在国内研究中尚属空白.本文分析了面板数据的数据格式和数字特征,根据聚类分析原理,重新构造了多指标面板数据的距离函数和离差平方和函数,在此基础上,说明了多指标面板数据的聚类分析过程.最后对我国各地区工业企业生产效率进行了聚类实证分析,显示了良好的效果。     

厚尾分布的极值分位数估计与极值风险测度研究

金融数据呈现的厚尾性已达成共识.本文中,我们基于指数回归模型构造了厚尾分布的极值分位数估计,从而得到了VaR的估计公式.作为一个应用,我们得到了上海上证指数和深圳成份指数的VaR的估计值.     

A Loss function approach to identifying environmental exceedances

A frequent problem in environmental science is the prediction of extrema and exceedances. It is well known that Bayesian and empirical-Bayesian predictors based on integrated squared error loss (ISEL) tend to overshrink predictions of extrema toward the mean. In this paper, we consider a geostatistical extension of the weighted rank squared error loss function (WRSEL) of Wright et al. (2003), which we call the integrated weighted quantile squared error loss (IWQSEL), as the basis for prediction of exceedances and their spatial location. The loss function is based on an ordering of the underlying spatial process using a spatially averaged cumulative distribution function. We illustrate this methodology with a Bayesian analysis of surface-nitrogen concentrations in the Chesapeake Bay and compare the new IWQSEL predictor with a standard ISEL predictor. We also give a comparison to predicted extrema obtained from a “plug-in” goestatistical analysis. AMS 2000 Subject Classification Primary—62M30; Secondary—62H11     

Low‐rank approximation of tensors via sparse optimization

The goal of this paper is to find a low‐rank approximation for a given nth tensor. Specifically, we give a computable strategy on calculating the rank of a given tensor, based on approximating the solution to an NP‐hard problem. In this paper, we formulate a sparse optimization problem via an l₁‐regularization to find a low‐rank approximation of tensors. To solve this sparse optimization problem, we propose a rescaling algorithm of the proximal alternating minimization and study the theoretical convergence of this algorithm. Furthermore, we discuss the probabilistic consistency of the sparsity result and suggest a way to choose the regularization parameter for practical computation. In the simulation experiments, the performance of our algorithm supports that our method provides an efficient estimate on the number of rank‐one tensor components in a given tensor. Moreover, this algorithm is also applied to surveillance videos for low‐rank approximation.     

Robust estimation in inverse problems via quantile coupling

In this article we consider a sequence of hierarchical space model of inverse problems.The underlying function is estimated from indirect observations over a variety of error distributions including those that are heavy-tailed and may not even possess variances or means.The main contribution of this paper is that we establish some oracle inequalities for the inverse problems by using quantile coupling technique that gives a tight bound for the quantile coupling between an arbitrary sample p-quantile and a normal variable,and an automatic selection principle for the nonrandom filters.This leads to the data-driven choice of weights.We also give an algorithm for its implementation.The quantile coupling inequality developed in this paper is of independent interest,because it includes the median coupling inequality in literature as a special case.     

Rank test for heteroscedastic functional data

In this paper, we consider (mid-)rank based inferences for testing hypotheses in a fully nonparametric marginal model for heteroscedastic functional data that contain a large number of within subject measurements from possibly only a limited number of subjects. The effects of several crossed factors and their interactions with time are considered. The results are obtained by establishing asymptotic equivalence between the rank statistics and their asymptotic rank transforms. The inference holds under the assumption ofα-mixing without moment assumptions. As a result, the proposed tests are applicable to data from heavy-tailed or skewed distributions, including both continuous and ordered categorical responses. Simulation results and a real application confirm that the (mid-)rank procedures provide both robustness and increased power over the methods based on original observations for non-normally distributed data.     

A new extreme quantile estimator for heavy-tailed distributions

The classical estimation method for extreme quantiles of heavy-tailed distributions was presented by Weissman (J. Amer. Statist. Assoc. 73 (1978) 812–815) and makes use of the Hill estimator (Ann. Statist. 3 (1975) 1163–1174) for the positive extreme value index. This index estimator can be interpreted as an estimator of the slope in the Pareto quantile plot in case one considers regression lines passing through a fixed anchor point. In this Note we propose a new extreme quantile estimator based on an unconstrained least squares estimator of the index, introduced by Kratz and Resnick (Comm. Statist. Stochastic Models 12 (1996) 699–724) and Schultze and Steinebach (Statist. Decisions 14 (1996) 353–372) and we study its asymptotic behavior. To cite this article: A. Fils, A. Guillou, C. R. Acad. Sci. Paris, Ser. I 338 (2004).     

Modelling conflicting information using subexponential distributions and related classes

In the Bayesian modelling the data and the prior information concerning a certain parameter of interest may conflict, in the sense that the information carried by them disagree. The most common form of conflict is the presence of outlying information in the data, which may potentially lead to wrong posterior conclusions. To prevent this problem we use robust models which aim to control the influence of the atypical information in the posterior distribution. Roughly speaking, we conveniently use heavy-tailed distributions in the model in order to resolve conflicts in favour of those sources of information which we believe is more credible. The class of heavy-tailed distributions is quite wide and the literature have been concerned in establishing conditions on the data and prior distributions in order to reject the outlying information. In this work we focus on the subexponential and $\mathfrak L $ classes of heavy-tailed distributions, in which we establish sufficient conditions under which the posterior distribution automatically rejects the conflicting information.     

On Singular Values Related to DAEs in Kronecker Canonical Form

The index and the structural properties of differential algebraic equations (DAEs) are often determined by rank considerations of the derivative array. Since the Kronecker canonical form is a well-understood standard form that permits deep insight into the properties of DAEs, in this contribution we undertake an analysis of the singular values of this specific derivative array. To this end, the special structure of the obtained block matrices is pointed out, such that some formulas for the computation and estimation of eigenvalues and singular values can be applied. Actually, we explore the relationship between the spectra of particular block tridiagonal matrices and some perturbed Jacobi matrices.

     

An augmented Lagrangian method for optimization problems with structured geometric constraints

This paper is devoted to the theoretical and numerical investigation of an augmented Lagrangian method for the solution of optimization problems with geometric constraints. Specifically, we study situations where parts of the constraints are nonconvex and possibly complicated, but allow for a fast computation of projections onto this nonconvex set. Typical problem classes which satisfy this requirement are optimization problems with disjunctive constraints (like complementarity or cardinality constraints) as well as optimization problems over sets of matrices which have to satisfy additional rank constraints. The key idea behind our method is to keep these complicated constraints explicitly in the constraints and to penalize only the remaining constraints by an augmented Lagrangian function. The resulting subproblems are then solved with the aid of a problem-tailored nonmonotone projected gradient method. The corresponding convergence theory allows for an inexact solution of these subproblems. Nevertheless, the overall algorithm computes so-called Mordukhovich-stationary points of the original problem under a mild asymptotic regularity condition, which is generally weaker than most of the respective available problem-tailored constraint qualifications. Extensive numerical experiments addressing complementarity- and cardinality-constrained optimization problems as well as a semidefinite reformulation of MAXCUT problems visualize the power of our approach.

     

在险风险值估计方法的比较研究

金融数据呈现的厚尾性已达成共识。本文首先基于指数回归模型提出了一种厚尾分布的极值分位数估计方法,得到了在险风险值的估计公式。然后得到了上海上证指数、国债指数和企业债券指数的在险风险值的估计值,比较了他们的极值风险.     

Error bound of critical points and KL property of exponent 1/2 for squared F-norm regularized factorization

This paper is concerned with the squared F(robenius)-norm regularized factorization form for noisy low-rank matrix recovery problems. Under a suitable assumption on the restricted condition number of the Hessian matrix of the loss function, we establish an error bound to the true matrix for the non-strict critical points with rank not more than that of the true matrix. Then, for the squared F-norm regularized factorized least squares loss function, we establish its KL property of exponent 1/2 on the global optimal solution set under the noisy and full sample setting, and achieve this property at its certain class of critical points under the noisy and partial sample setting. These theoretical findings are also confirmed by solving the squared F-norm regularized factorization problem with an accelerated alternating minimization method.

     

Quantile estimations via modified Cholesky decomposition for longitudinal single-index models

Quantile regression is a powerful complement to the usual mean regression and becomes increasingly popular due to its desirable properties. In longitudinal studies, it is necessary to consider the intra-subject correlation among repeated measures over time to improve the estimation efficiency. In this paper, we focus on longitudinal single-index models. Firstly, we apply the modified Cholesky decomposition to parameterize the intra-subject covariance matrix and develop a regression approach to estimate the parameters of the covariance matrix. Secondly, we propose efficient quantile estimating equations for the index coefficients and the link function based on the estimated covariance matrix. Since the proposed estimating equations include a discrete indicator function, we propose smoothed estimating equations for fast and accurate computation of the index coefficients, as well as their asymptotic covariances. Thirdly, we establish the asymptotic properties of the proposed estimators. Finally, simulation studies and a real data analysis have illustrated the efficiency of the proposed approach.

     

Unified Noncrossing Multiple Quantile Regressions Tree

In this article, we consider the estimation problem of a tree model for multiple conditional quantile functions of the response. Using the generalized, unbiased interaction detection and estimation algorithm, the quantile regression tree (QRT) method has been developed to construct a tree model for an individual quantile function. However, QRT produces different tree models across quantile levels because it estimates several QRT models separately. Furthermore, the estimated quantile functions from QRT often cross each other and consequently violate the basic properties of quantiles. This undesirable phenomenon reduces prediction accuracy and makes it difficult to interpret the resulting tree models. To overcome such limitations, we propose the unified noncrossing multiple quantile regressions tree (UNQRT) method, which constructs a common tree structure across all interesting quantile levels for better data visualization and model interpretation. Furthermore, the UNQRT estimates noncrossing multiple quantile functions simultaneously by enforcing noncrossing constraints, resulting in the improvement of prediction accuracy. The numerical results are presented to demonstrate the competitive performance of the proposed UNQRT over QRT. Supplementary materials for this article are available online.     

无交叉多指标分位数回归模型中的估计与推断

在过去的30年中分位数回归模型的研究已十分深入.然而在实际的应用场景中,由传统估计方法所得到的分位数回归估计量,经常会在不同分位数水平上出现互相交叉的现象,这给分位数回归模型的实际应用造成了解释和预测上的困难.为解决这个问题,本文提出一种带单调约束的半参数多指标分位数回归模型的研究框架.首先将半参数多指标分位数回归模型...     

On directional multiple-output quantile regression

This paper sheds some new light on projection quantiles. Contrary to the sophisticated set analysis used in Kong and Mizera (2008) [13], we adopt a more parametric approach and study the subgradient conditions associated with these quantiles. In this setup, we introduce Lagrange multipliers which can be interpreted in various interesting ways, in particular in a portfolio optimization context. The corresponding projection quantile regions were already shown to coincide with the halfspace depth ones in Kong and Mizera (2008) [13], but we provide here an alternative proof (completely based on projection quantiles) that has the advantage of leading to an exact computation of halfspace depth regions from projection quantiles. Above all, we systematically consider the regression case, which was barely touched in Kong and Mizera (2008) [13]. We show in particular that the regression quantile regions introduced in Hallin, Paindaveine, and Šiman (2010) [6] and [7] can also be obtained from projection (regression) quantiles, which may lead to a faster computation of those regions in some particular cases.     

p-Adic Analogue of the Wave Equation

In this paper, a p-adic analogue of the wave equation with Lipschitz source is considered. Since it is hard to arrive the solution of the problem, we propose a regularized method to solve the problem from a modified p-adic integral equation. Moreover, we give an iterative scheme for numerical computation of the regularlized solution.