Estimating a reduced rank regression model for non-normal variables

A method using third order moments for estimating the regression coefficients as well as the latent state scores of the reduced-rank regression model when the latent variable(s) are non-normally distributed is presented in this paper. It is shown that the factor analysis type indeterminacy of the regression coefficient matrices is eliminated. A real life example of the proposed method is presented. Differences of this solution with the reduced-rank regression eigen solution are discussed.     

Robust model selection in regression

A robust version of Akaike's model selection procedure for regression models is introduced and its relationship with robust testing procedures is discussed.     

Robust nonparametric regression estimation

In this paper we define a robust conditional location functional without requiring any moment condition. We apply the nonparametric proposals considered by C. Stone (Ann. Statist. 5 (1977), 595–645) to this functional equation in order to obtain strongly consistent, robust nonparametric estimates of the regression function. We give some examples by using nearest neighbor weights or weights based on kernel methods under no assumptions whatsoever on the probability measure of the vector (X,Y). We also derive strong convergence rates and the asymptotic distribution of the proposed estimates.     

Robust regression function estimation

A robust estimator of the regression function is proposed combining kernel methods as introduced for density estimation and robust location estimation techniques. Weak and strong consistency and asymptotic normality are shown under mild conditions on the kernel sequence. The asymptotic variance is a product from a factor depending only on the kernel and a factor similar to the asymptotic variance in robust estimation of location. The estimation is minimax robust in the sense of Huber (1964). Robust estimation of a location parameter. Ann. Math. Statist.33 73–101.     

Dimensionality reduction and volume minimization—generalization of the determinant minimization criterion for reduced rank regression problems

In this article we propose a generalization of the determinant minimization criterion. The problem of minimizing the determinant of a matrix expression has implicit assumptions that the objective matrix is always nonsingular. In case of singular objective matrix the determinant would be zero and the minimization problem would be meaningless. To be able to handle all possible cases we generalize the determinant criterion to rank reduction and volume minimization of the objective matrix. The generalized minimization criterion is used to solve the following ordinary reduced rank regression problem:

min_rank(X)=kdet(B-XA)(B-XA)^T,     

Nonlinear Schwarz iterations with reduced rank extrapolation

Extrapolation methods can be a very effective technique used for accelerating the convergence of vector sequences. In this paper, these methods are used to accelerate the convergence of Schwarz iterative methods for nonlinear problems. A new implementation of the reduced-rank-extrapolation (RRE) method is introduced. Some convergence analysis is presented, and it is shown numerically that certain extrapolation methods can indeed be very effective in accelerating the convergence of Schwarz methods.     

On the weighted multivariate Wilcoxon rank regression estimate

Zhou (2010) introduced a multivariate Wilcoxon regression estimate which possesses some nice properties: computational ease, asymptotic normality and high efficiency. However, it is sensitive to the leverage points. To circumvent this problem, we propose a weighted multivariate Wilcoxon regression estimate. Under some regularity conditions, the asymptotic normality is established. We further study the robustness of the proposed estimate through the influence function. By properly choosing the weight functions, our results show that the corresponding estimate can have bounded influence function on both response and covariates.     

Kolmogorov-Smirnov two-sample test based on regression rank scores

We derive the two-sample Kolmogorov-Smirnov type test when a nuisance linear regression is present. The test is based on regression rank scores and provides a natural extension of the classical Kolmogorov-Smirnov test. Its asymptotic distributions under the hypothesis and the local alternatives coincide with those of the classical test. (Supplement to the special issue of Appl. Math. 53 (2008), No. 3) This work was supported by the Czech Science Foundation under Grant No. 201/05/H007 and by Research Project LC06024.     

Asymptotically most powerful rank tests for regression parameters in manova

Summary Srivastava [5] proposed a class of rank score tests for testing the hypothesis that β₁=⋯β _p =0 in the linear regression modely _i =β₁ x _1i +β₂ x _2i +⋯+β _p +x _pi +ɛ _i under weaker conditions than Hájek [2]. In this paper, under the same weak conditions, a class of rank score tests is proposed for testing β₁=⋯β _q =0 in the multivariate linear regression modely _i =β₁ x _1i +β₂ x _2i +⋯+β _p +x _pi +ɛ _i ,q≦p, where β _i ’s arek-vectors. The limiting distribution of the test statistic is shown to be central χ _qk ² underH and non-central χ _qk ² under a sequence of alternatives tending to the hypothesis at a suitable rate. Research supported by Canada Council and National Research Council of Canada.     

Robust estimation for nonparametric generalized regression

This paper focuses on nonparametric regression estimation for the parameters of a discrete or continuous distribution, such as the Poisson or Gamma distributions, when anomalous data are present. The proposal is a natural extension of robust methods developed in the setting of parametric generalized linear models. Robust estimators bounding either large values of the deviance or of the Pearson residuals are introduced and their asymptotic behaviour is derived. Through a Monte Carlo study, for the Poisson and Gamma distributions, the finite properties of the proposed procedures are investigated and their performance is compared with that of the classical ones. A resistant cross-validation method to choose the smoothing parameter is also considered.     

On estimation in some reduced rank extended growth curve models

The general multivariate analysis of variance model has been extensively studied in the statistical literature and successfully applied in many different fields for analyzing longitudinal data. In this article, we consider the extension of this model having two sets of regressors constituting a growth curve portion and a multivariate analysis of variance portion, respectively. Nowadays, the data collected in empirical studies have relatively complex structures though often demanding a parsimonious modeling. This can be achieved for example through imposing rank constraints on the regression coefficient matrices. The reduced rank regression structure also provides a theoretical interpretation in terms of latent variables. We derive likelihood based estimators for the mean parameters and covariance matrix in this type of models. A numerical example is provided to illustrate the obtained results.     

Robust weighted orthogonal regression in the errors-in-variables model

This paper focuses on robust estimation in the structural errors-in-variables (EV) model. A new class of robust estimators, called weighted orthogonal regression estimators, is introduced. Robust estimators of the parameters of the EV model are simply derived from robust estimators of multivariate location and scatter such as the M-estimators, the S-estimators and the MCD estimator. The influence functions of the proposed estimators are calculated and shown to be bounded. Moreover, we derive the asymptotic distributions of the estimators and illustrate the results on simulated examples and on a real-data set.     

Robust kernel ridge regression based on M-estimation

Ridge regression (RR) and kernel ridge regression (KRR) are important tools to avoid the effects of multicollinearity. However, the predictions of RR and KRR become inappropriate for use in regression models when data are contaminated by outliers. In this paper, we propose an algorithm to obtain a nonlinear robust prediction without specifying a nonlinear model in advance. We combine M-estimation and kernel ridge regression to obtain the nonlinear prediction. Then, we compare the proposed method with some other methods.     

Minimal polynomial and reduced rank extrapolation methods are related

Minimal Polynomial Extrapolation (MPE) and Reduced Rank Extrapolation (RRE) are two polynomial methods used for accelerating the convergence of sequences of vectors {x _m }. They are applied successfully in conjunction with fixed-point iterative schemes in the solution of large and sparse systems of linear and nonlinear equations in different disciplines of science and engineering. Both methods produce approximations s _k to the limit or antilimit of {x _m } that are of the form \(\boldsymbol {s}_{k}={\sum }^{k}_{i=0}\gamma _{i}\boldsymbol {x}_{i}\) with \({\sum }^{k}_{i=0}\gamma _{i}=1\), for some scalars γ _i . The way the two methods are derived suggests that they might, somehow, be related to each other; this has not been explored so far, however. In this work, we tackle this issue and show that the vectors \(\boldsymbol {s}_{k}^{\textit {{\tiny {MPE}}}}\) and \(\boldsymbol {s}_{k}^{\textit {{\tiny {RRE}}}}\) produced by the two methods are related in more than one way, and independently of the way the x _m are generated. One of our results states that RRE stagnates, in the sense that \(\boldsymbol {s}_{k}^{\textit {{\tiny {RRE}}}}=\boldsymbol {s}_{k-1}^{\textit {{\tiny {RRE}}}}\), if and only if \(\boldsymbol {s}_{k}^{\textit {{\tiny {MPE}}}}\) does not exist. Another result states that, when \(\boldsymbol {s}_{k}^{\textit {{\tiny {MPE}}}}\) exists, there holds

$$\mu_{k}\boldsymbol{s}_{k}^{\textit{{\tiny{RRE}}}}=\mu_{k-1}\boldsymbol{s}_{k-1}^{\textit{{\tiny{RRE}}}}+ \nu_{k}\boldsymbol{s}_{k}^{\textit{{\tiny{MPE}}}}\quad \text{with}\quad \mu_{k}=\mu_{k-1}+\nu_{k}, $$

for some positive scalars μ _k , μ _k?1, and ν _k that depend only on \(\boldsymbol {s}_{k}^{\textit {{\tiny {RRE}}}}\), \(\boldsymbol {s}_{k-1}^{\textit {{\tiny {RRE}}}}\), and \(\boldsymbol {s}_{k}^{\textit {{\tiny {MPE}}}}\), respectively. Our results are valid when MPE and RRE are defined in any weighted inner product and the norm induced by it. They also contain as special cases the known results pertaining to the connection between the method of Arnoldi and the method of generalized minimal residuals, two important Krylov subspace methods for solving nonsingular linear systems.

     

Privacy preserving linear regression modeling of distributed databases

Statistical analysis is one of the important tools in data mining field. Little work has been conducted to investigate how statistical analysis could be performed when dataset are distributed among a number of data owners. Due to confidentiality or other proprietary reasons, data owners are reluctant to share data with others, while they wish to perform statistical analysis cooperatively. We address the important tradeoff between privacy and global statistical analysis such as linear regression, and present a privacy preserving linear regression model based on fully homomorphic encryption scheme.     

Robust regression credibility: The influence function approach

In classical credibility theory we assume that the vector of claims conditionally on has independent components with identical means. However, this assumption is sometimes unrealistic. To relax this condition Hachemeister (Hachemeister, C.A., 1975. Credibility for regression models with application to trend. In: Kahn, P. (Ed.), Credibility, Theory and Applications. Academic Press, New York) introduced regressors. The presence of large claims can perturb the credibility premium estimation. The lack of robustness of regression credibility estimators, as well as the fairness of tariff evaluation, led to the development of this paper. Our proposal is to apply robust statistics to the regression credibility estimation by using the robust influence function approach of M-estimators.     

ELECTRE: Robust ordinal regression for outranking methods

We present a new method, called ELECTRE^GKMS, which employs robust ordinal regression to construct a set of outranking models compatible with preference information. The preference information supplied by the decision maker (DM) is composed of pairwise comparisons stating the truth or falsity of the outranking relation for some real or fictitious reference alternatives. Moreover, the DM specifies some ranges of variation of comparison thresholds on considered pseudo-criteria. Using robust ordinal regression, the method builds a set of values of concordance indices, concordance thresholds, indifference, preference, and veto thresholds, for which all specified pairwise comparisons can be restored. Such sets are called compatible outranking models. Using these models, two outranking relations are defined, necessary and possible. Whether for an ordered pair of alternatives there is necessary or possible outranking depends on the truth of outranking relation for all or at least one compatible model, respectively. Distinguishing the most certain recommendation worked out by the necessary outranking, and a possible recommendation worked out by the possible outranking, ELECTRE^GKMS answers questions of robustness concern. The method is intended to be used interactively with incremental specification of pairwise comparisons, possibly with decreasing confidence levels. In this way, the necessary and possible outranking relations can be, respectively, enriched or impoverished with the growth of the number of pairwise comparisons. Furthermore, the method is able to identify troublesome pieces of preference information which are responsible for incompatibility. The necessary and possible outranking relations are to be exploited as usual outranking relations to work out recommendation in choice or ranking problems. The introduced approach is illustrated by a didactic example showing how ELECTRE^GKMS can support real-world decision problems.     

Multi-round smoothed composite quantile regression for distributed data

Annals of the Institute of Statistical Mathematics - Statistical analysis of large-scale dataset is challenging due to the limited memory constraint and computation source and calls for the...     

Robust logistic zero-sum regression for microbiome compositional data

Advances in Data Analysis and Classification - We introduce the Robust Logistic Zero-Sum Regression (RobLZS) estimator, which can be used for a two-class problem with high-dimensional compositional...