Characterizations of multivariate normality. I. Through independence of some statistics

It is established that a vector variable (X₁, …, X_k) has a multivariate normal distribution if for each X_i the regression on the rest is linear and the conditional distribution about the regression does not depend on the rest of the variables, provided the regression coefficients satisfy some mild conditions. The result is extended to the case where X_i themselves are vector variables.     

Exact and approximation methods for dependability assessment of tram systems with time window

The transportation system examined in this paper is the city tram one, where failed trams are replaced by reliable spare ones. If failed tram is repaired and delivered, then it comes back on work. There is the time window that failed tram has to be either replaced (exchanged) by spare or by repaired and delivered within. Time window is therefore paramount to user perception of transport system unreliability. Time between two subsequent failures, exchange time, and repair together with delivery time, respectively, are described by random variables A, E, and D. A/E/D is selected as the notation for these random variables. There is a finite number of spare trams. Delivery time does not depend on the number of repair facilities. Hence, repair and delivery process can be treated as one with infinite number of facilities. Undesirable event called hazard is the event: neither the replacement nor the delivery has been completed in the time window. The goal of the paper is to find the following relationships: hazard probability of the tram system and mean hazard time as functions of number of spare trams. For systems with exponential time between failures, Weibull exchange and exponential delivery (so M/W/M in the proposed notation) two accurate solutions have been found. For systems with Weibull time between failures with shape in the range from 0.9 to 1.1, Weibull exchange and exponential delivery (i.e. W/W/M) a method yielding small errors has been provided. For the most general and difficult case in which all the random variables conform to Weibull distribution (W/W/W) a method returning moderate errors has been given.     

Correlation and the time interval in multiple regression models

In this paper we investigate the time interval effect of multiple regression models in which some of the variables are additive and some are multiplicative. The effect on the partial regression and correlation coefficients is influenced by the selected time interval. We find that the partial regression and correlation coefficients between two additive variables approach one-period values as n increases. When one of the variables is multiplicative, they will approach zero in the limit. We also show that the decreasing speed of the n-period correlation coefficients between both multiplicative variables is faster than others, except that a one-period correlation has a higher positive value. The results of this paper can be widely applied in various fields where regression or correlation analyses are employed.     

Promoting symmetric weight selection in data envelopment analysis: A penalty function approach

Traditionally, data envelopment analysis models assume total flexibility in weight selection, though this assumption can lead to several variables being ignored in determining the efficiency score. Existing methods constrain weight selection to a predefined range, thus removing possible feasible solutions. As such, in this paper we propose the symmetric weight assignment technique (SWAT) that does not affect feasibility and rewards decision making units (DMUs) that make a symmetric selection of weights. This allows for a method of weight restrictions that does not require preference constraints on the variables. Moreover, we show that the SWAT method may be used to differentiate among efficient DMUs.     

Partial least squares classification for high dimensional data using the PCOUT algorithm

Classification of samples into two or multi-classes is to interest of scientists in almost every field. Traditional statistical methodology for classification does not work well when there are more variables (p) than there are samples (n) and it is highly sensitive to outlying observations. In this study, a robust partial least squares based classification method is proposed to handle data containing outliers where $n\ll p.$ The proposed method is applied to well-known benchmark datasets and its properties are explored by an extensive simulation study.     

Logical laws for existential monadic second-order sentences with infinite first-order parts

We consider existential monadic second-order sentences ?X φ(X) about undirected graphs, where ?X is a finite sequence of monadic quantifiers and φ(X) ∈ +_∞ω^ω is an infinite first-order formula. We prove that there exists a sentence (in the considered logic) with two monadic variables and two first-order variables such that the probability that it is true on G(n, p) does not converge. Moreover, such an example is also obtained for one monadic variable and three first-order variables.     

The balanced linear programming problem

The Balanced Linear Programming Problem (BLPP) arises in situations which require equitable distribution of a scarce resource. The BLPP can be transformed to the standard form of the linear programming problem by introducing 2∥N∥ + 2 additional variables and 2∥N∥ additional constraints. This transformation is not desirable from the computational point of view for larger values of ∥N∥ as it increases the problem size substantially. It is also undesirable from a theoretical perspective as it might affect the special structure of the constraint matrix. In this paper, we develop an algorithm for the BLPP which does not require problem enlargement. The algorithm is based on the relationship between the BLPP and the minimax linear programming problem, and solving the latter problem parametrically. Our algorithm, in essence, performs steps that are similar to those performed in the parametric simplex method with parametric right hand side. We then adapt our algorithm for the network flow problem and this specialized algorithm can be applied on the network directly without maintaining the simplex tableau.     

Random weights, robust lattice rules and the geometry of the cbcrc algorithm

In this paper we study lattice rules which are cubature formulae to approximate integrands over the unit cube [0,1] ^s from a weighted reproducing kernel Hilbert space. We assume that the weights are independent random variables with a given mean and variance for two reasons stemming from practical applications: (i) It is usually not known in practice how to choose the weights. Thus by assuming that the weights are random variables, we obtain robust constructions (with respect to the weights) of lattice rules. This, to some extend, removes the necessity to carefully choose the weights. (ii) In practice it is convenient to use the same lattice rule for many different integrands. The best choice of weights for each integrand may vary to some degree, hence considering the weights random variables does justice to how lattice rules are used in applications. In this paper the worst-case error is therefore a random variable depending on random weights. We show how one can construct lattice rules which perform well for weights taken from a set with large measure. Such lattice rules are therefore robust with respect to certain changes in the weights. The construction algorithm uses the component-by-component (cbc) idea based on two criteria, one using the mean of the worst case error and the second criterion using a bound on the variance of the worst-case error. We call the new algorithm the cbc2c (component-by-component with 2 constraints) algorithm. We also study a generalized version which uses r constraints which we call the cbcrc (component-by-component with r constraints) algorithm. We show that lattice rules generated by the cbcrc algorithm simultaneously work well for all weights in a subspace spanned by the chosen weights ?? ⁽¹⁾, . . . , ?? ^(r). Thus, in applications, instead of finding one set of weights, it is enough to find a convex polytope in which the optimal weights lie. The price for this method is a factor r in the upper bound on the error and in the construction cost of the lattice rule. Thus the burden of determining one set of weights very precisely can be shifted to the construction of good lattice rules. Numerical results indicate the benefit of using the cbc2c algorithm for certain choices of weights.     

An improvement in DS-BiCGstab(l) and its application for linear systems in lattice QCD

An improved algorithm of the DS-BiCGstab(l) method (IDSL) is proposed. The DS-BiCGstab(l) method (DSL) selects the value of l dynamically for the BiCGstab(l) method, but it does not always select a proper value for l in terms of its convergence. From this viewpoint, this research has focused on converging behavior with the BiCGstab(l) method, and the DSL has been improved. Several numerical results in lattice QCD show that this improvement leads dynamic selection of l to a value relevant to the convergence than the DSL without a substantial increase in the calculating costs.     

On the dependence structure of order statistics

Given a random sample from a continuous variable, it is observed that the copula linking any pair of order statistics is independent of the parent distribution. To compare the degree of association between two such pairs of ordered random variables, a notion of relative monotone regression dependence (or stochastic increasingness) is considered. Using this concept, it is proved that for i<j, the dependence of the jth order statistic on the ith order statistic decreases as i and j draw apart. This extends earlier results of Tukey (Ann. Math. Statist. 29 (1958) 588) and Kim and David (J. Statist. Plann. Inference 24 (1990) 363). The effect of the sample size on this type of dependence is also investigated, and an explicit expression is given for the population value of Kendall's coefficient of concordance between two arbitrary order statistics of a random sample.     

Ordinal regression revisited: Multiple criteria ranking using a set of additive value functions

We present a new method, called UTA^GMS, for multiple criteria ranking of alternatives from set A using a set of additive value functions which result from an ordinal regression. The preference information provided by the decision maker is a set of pairwise comparisons on a subset of alternatives A^R ⊆ A, called reference alternatives. The preference model built via ordinal regression is the set of all additive value functions compatible with the preference information. Using this model, one can define two relations in the set A: the necessary weak preference relation which holds for any two alternatives a, b from set A if and only if for all compatible value functions a is preferred to b, and the possible weak preference relation which holds for this pair if and only if for at least one compatible value function a is preferred to b. These relations establish a necessary and a possible ranking of alternatives from A, being, respectively, a partial preorder and a strongly complete relation. The UTA^GMS method is intended to be used interactively, with an increasing subset A^R and a progressive statement of pairwise comparisons. When no preference information is provided, the necessary weak preference relation is a weak dominance relation, and the possible weak preference relation is a complete relation. Every new pairwise comparison of reference alternatives, for which the dominance relation does not hold, is enriching the necessary relation and it is impoverishing the possible relation, so that they converge with the growth of the preference information. Distinguishing necessary and possible consequences of preference information on the complete set of actions, UTA^GMS answers questions of robustness analysis. Moreover, the method can support the decision maker when his/her preference statements cannot be represented in terms of an additive value function. The method is illustrated by an example solved using the UTA^GMS software. Some extensions of the method are also presented.     

A unified approach for finding real and integer solutions to systems of linear inequalities

The variable-elimination method for solving linear inequalities is used for finding integer solutions. Properties of this method enable one to give a simple proof, for a large class of systems of linear inequalities, that if a system in this class has any (real-valued) solution, then it also has an integer solution. This class includes all systems of the form Ax>b where A is any real matrix and b does not contain any negative element. The variable-elimination method has an exponential bound on the storage requirement, and hence on the execution time. These exists a simple strategy aimed at reducing the amount of storage and execution time needed. An experimental implementation was used to explore the effectiveness of this strategy.     

Fast algorithms for robust classification with Bayesian nets

We focus on a well-known classification task with expert systems based on Bayesian networks: predicting the state of a target variable given an incomplete observation of the other variables in the network, i.e., an observation of a subset of all the possible variables. To provide conclusions robust to near-ignorance about the process that prevents some of the variables from being observed, it has recently been derived a new rule, called conservative updating. With this paper we address the problem to efficiently compute the conservative updating rule for robust classification with Bayesian networks. We show first that the general problem is NP-hard, thus establishing a fundamental limit to the possibility to do robust classification efficiently. Then we define a wide subclass of Bayesian networks that does admit efficient computation. We show this by developing a new classification algorithm for such a class, which extends substantially the limits of efficient computation with respect to the previously existing algorithm. The algorithm is formulated as a variable elimination procedure, whose computation time is linear in the input size.     

Robust kernel-based regression with bounded influence for outliers

The kernel-based regression (KBR) method, such as support vector machine for regression (SVR) is a well-established methodology for estimating the nonlinear functional relationship between the response variable and predictor variables. KBR methods can be very sensitive to influential observations that in turn have a noticeable impact on the model coefficients. The robustness of KBR methods has recently been the subject of wide-scale investigations with the aim of obtaining a regression estimator insensitive to outlying observations. However, existing robust KBR (RKBR) methods only consider Y-space outliers and, consequently, are sensitive to X-space outliers. As a result, even a single anomalous outlying observation in X-space may greatly affect the estimator. In order to resolve this issue, we propose a new RKBR method that gives reliable result even if a training data set is contaminated with both Y-space and X-space outliers. The proposed method utilizes a weighting scheme based on the hat matrix that resembles the generalized M-estimator (GM-estimator) of conventional robust linear analysis. The diagonal elements of hat matrix in kernel-induced feature space are used as leverage measures to downweight the effects of potential X-space outliers. We show that the kernelized hat diagonal elements can be obtained via eigen decomposition of the kernel matrix. The regularized version of kernelized hat diagonal elements is also proposed to deal with the case of the kernel matrix having full rank where the kernelized hat diagonal elements are not suitable for leverage. We have shown that two kernelized leverage measures, namely, the kernel hat diagonal element and the regularized one, are related to statistical distance measures in the feature space. We also develop an efficiently kernelized training algorithm for the parameter estimation based on iteratively reweighted least squares (IRLS) method. The experimental results from simulated examples and real data sets demonstrate the robustness of our proposed method compared with conventional approaches.     

Convergence de l'estimateur à noyau des k plus proches voisins en régression fonctionnelle non-paramétrique

This note focuses on the k nearest neighbor method when one regresses a real random variable on a functional random variable (i.e. valued in an infinite-dimensional space). More precisely, we consider a kernel estimator of the regression based on a local bandwidth using exactly the k nearest neighbors. Although it is frequently used in functional data analysis, this method has not given any theoretical result so far. The aim of this Note is to show the pointwise almost-complete convergence of the k nearest neighbor kernel estimator in nonparametric functional regression. To cite this article: F. Burba et al., C. R. Acad. Sci. Paris, Ser. I 346 (2008).     

A variable selection method based on Tabu search for logistic regression models

A Tabu search method is proposed and analysed for selecting variables that are subsequently used in Logistic Regression Models. The aim is to find from among a set of m variables a smaller subset which enables the efficient classification of cases. Reducing dimensionality has some very well-known advantages that are summarized in literature. The specific problem consists in finding, for a small integer value of p, a subset of size p of the original set of variables that yields the greatest percentage of hits in Logistic Regression. The proposed Tabu search method performs a deep search in the solution space that alternates between a basic phase (that uses simple moves) and a diversification phase (to explore regions not previously visited). Testing shows that it obtains significantly better results than the Stepwise, Backward or Forward methods used by classic statistical packages. Some results of applying these methods are presented.     

On the implementation of LIR: the case of simple linear regression with interval data

This paper considers the problem of simple linear regression with interval-censored data. That is, \(n\) pairs of intervals are observed instead of the \(n\) pairs of precise values for the two variables (dependent and independent). Each of these intervals is closed but possibly unbounded, and contains the corresponding (unobserved) value of the dependent or independent variable. The goal of the regression is to describe the relationship between (the precise values of) these two variables by means of a linear function. Likelihood-based Imprecise Regression (LIR) is a recently introduced, very general approach to regression for imprecisely observed quantities. The result of a LIR analysis is in general set-valued: it consists of all regression functions that cannot be excluded on the basis of likelihood inference. These regression functions are said to be undominated. Since the interval data can be unbounded, a robust regression method is necessary. Hence, we consider the robust LIR method based on the minimization of the residuals’ quantiles. For this method, we prove that the set of all the intercept-slope pairs corresponding to the undominated regression functions is the union of finitely many polygons. We give an exact algorithm for determining this set (i.e., for determining the set-valued result of the robust LIR analysis), and show that it has worst-case time complexity \(O(n^{3}\log n)\) . We have implemented this exact algorithm as part of the R package linLIR.     

Selecting rows and columns for training support vector regression models with large retail datasets

Although support vector regression models are being used successfully in various applications, the size of the business datasets with millions of observations and thousands of variables makes training them difficult, if not impossible to solve. This paper introduces the Row and Column Selection Algorithm (ROCSA) to select a small but informative dataset for training support vector regression models with standard SVM tools. ROCSA uses ε-SVR models with L₁-norm regularization of the dual and primal variables for the row and column selection steps, respectively. The first step involves parallel processing of data chunks and selects a fraction of the original observations that are either representative of the pattern identified in the chunk, or represent those observations that do not fit the identified pattern. The column selection step dramatically reduces the number of variables and the multicolinearity in the dataset, increasing the interpretability of the resulting models and their ease of maintenance. Evaluated on six retail datasets from two countries and a publicly available research dataset, the reduced ROCSA training data improves the predictive accuracy on average by 39% compared with the original dataset when trained with standard SVM tools. Comparison with the ε SSVR method using reduced kernel technique shows similar performance improvement. Training a standard SVM tool with the ROCSA selected observations improves the predictive accuracy on average by 21% compared to the practical approach of random sampling.     

A bias corrected nonparametric regression estimator

In this article, we propose a new method of bias reduction in nonparametric regression estimation. The proposed new estimator has asymptotic bias order h⁴, where h is a smoothing parameter, in contrast to the usual bias order h² for the local linear regression. In addition, the proposed estimator has the same order of the asymptotic variance as the local linear regression. Our proposed method is closely related to the bias reduction method for kernel density estimation proposed by Chung and Lindsay (2011). However, our method is not a direct extension of their density estimate, but a totally new one based on the bias cancelation result of their proof.     

Characterizations of multivariate normality II. Through linear regressions

It is established that a vector (X′₁, X′₂, …, X′_k) has a multivariate normal distribution if (i) for each X_i the regression on the rest is linear, (ii) the conditional distribution of X₁ about the regression does not depend on the rest of the variables, and (iii) the conditional distribution of X₂ about the regression does not depend on the rest of the variables, provided that the regression coefficients satisfy some more conditions that those given by [4]J. Multivar. Anal. 6 81–94].