Exact Misclassification Probabilities for Plug-In Normal Quadratic Discriminant Functions: II. The Heterogeneous Case

We consider the problem of discriminating between two independent multivariate normal populations, N_p(μ₁, Σ₁) and N_p(μ₂, Σ₂), having distinct mean vectors μ₁ and μ₂ and distinct covariance matrices Σ₁ and Σ₂. The parameters μ₁, μ₂, Σ₁, and Σ₂ are unknown and are estimated by means of independent random training samples from each population. We derive a stochastic representation for the exact distribution of the “plug-in” quadratic discriminant function for classifying a new observation between the two populations. The stochastic representation involves only the classical standard normal, chi-square, and F distributions and is easily implemented for simulation purposes. Using Monte Carlo simulation of the stochastic representation we provide applications to the estimation of misclassification probabilities for the well-known iris data studied by Fisher (Ann. Eugen.7 (1936), 179–188); a data set on corporate financial ratios provided by Johnson and Wichern (Applied Multivariate Statistical Analysis, 4th ed., Prentice–Hall, Englewood Cliffs, NJ, 1998); and a data set analyzed by Reaven and Miller (Diabetologia16 (1979), 17–24) in a classification of diabetic status.     

More on Stochastic Comparisons and Dependence among Concomitants of Order Statistics

For a sample of iid observations {(X_i, Y_i)} from an absolutely continuous distribution, the multivariate dependence of concomitants Y_[]=(Y_[1], Y_[2], …, Y_[n]) and the stochastic order of subsets of Y_[] are studied. If (X, Y) is totally positive dependent of order 2, Y_[] is multivariate totally positive dependent of order 2. If the conditional hazard rate function of Y given X, h_YX(yx), is decreasing in x for every y, Y_[] is multivariate right corner set increasing. And if Y is stochastically increasing in X, the concomitants are increasing in multivariate stochastic order.     

Classes of orderings of measures and related correlation inequalities II. Multivariate reverse rule distributions

A function f(x) defined on = ₁ × ₂ × … × _n where each _i is totally ordered satisfying f(x y) f(x y) ≥ f(x) f(y), where the lattice operations and refer to the usual ordering on , is said to be multivariate totally positive of order 2 (MTP₂). A random vector Z = (Z₁, Z₂,…, Z_n) of n-real components is MTP₂ if its density is MTP₂. Classes of examples include independent random variables, absolute value multinormal whose covariance matrix Σ satisfies −DΣ⁻¹D with nonnegative off-diagonal elements for some diagonal matrix D, characteristic roots of random Wishart matrices, multivariate logistic, gamma and F distributions, and others. Composition and marginal operations preserve the MTP₂ properties. The MTP₂ property facilitate the characterization of bounds for confidence sets, the calculation of coverage probabilities, securing estimates of multivariate ranking, in establishing a hierarchy of correlation inequalities, and in studying monotone Markov processes. Extensions on the theory of MTP₂ kernels are presented and amplified by a wide variety of applications.     

Estimation of multivariate binary density using orthogonal functions

In this paper, the authors studied certain properties of the estimate of Liang and Krishnaiah (1985, J. Multivariate Anal. 16, 162–172) for multivariate binary density. An alternative shrinkage estimate is also obtained. The above results are generalized to general orthonormal systems.     

Sequential Estimation of the Mean Vector of a Multivariate Linear Process

Sequential procedures are proposed to estimate the unknown mean vector of a multivariate linear process of the form X_t − μ = ∑^∞_{j = 0}A_jZ_{t − j}, where the Z_t are i.i.d. (0, Σ) with unknown covariance matrix Σ. The proposed point estimation is asymptotically risk efficient in the sense of Starr (1966, Ann. Math. Statist.37 1173-1185). The fixed accuracy confidence set procedure is asymptotically efficient with prescribed coverage probability in the sense of Chow and Robbins (1965, Ann. Math. Statist.36 457-462). A random central limit theorem for this process, under a mild summability condition on the coefficient matrices A_j, is also obtained.     

On Domains of Attraction of Multivariate Extreme Value Distributions under Absolute Continuity

The paper gives sufficient conditions for domains of attraction of multivariate extreme value distributions. Under the assumption of absolute continuity of a multivariate distribution, the criteria enable one to examine, by using limits of some rescaled conditional densities, whether the distribution belongs to the domain of attraction of some multivariate extreme value distribution. If this is the case, the criteria also determine how to construct such an extreme value distribution. Unlike the criterion given by de Haan and Resnick [1987,Stochastic Process. Appl.2583–93], the criteria are easily applicable even when the marginal tails are not Pareto-like.     

On the cumulants of affine equivariant estimators in elliptical families

Given a statistical model for data which take values in R^d and have elliptically distributed errors, and affine equivariant estimators and of a mean vector in R^dRⁿ and a d × d scatter matrix, expressions are given for the covarances of the estimators in terms of their expectations and some unknown constants that depend on the model and the estimator. Higher order cumulants are also developed. These results place considerable constraints on the possible cumulants of and , as wel as those of estimators of higher order behavior such as multivariate skewness and kurtosis. These expressions are obtained using tensor methods.     

Noninformative Priors for Multivariate Linear Calibration

This paper derives a class of first order probability matching priors and a complete catalog of the reference priors for the general multivariate linear calibration problem. In an important special case, a complete characterization of first order probability matching priors is given, and a fairly general class of second order probability matching priors is also provided. Orthogonal transformations (1987, D. R. Cox and N. Reid, J. Roy. Statist. Soc. Ser. B49, 1–18) are found to facilitate the derivations. It turns out that under orthogonal parameterization, reference priors (including Jeffreys' prior) are first order probability matching priors for unidimensional multivariate linear calibration. Also, in univariate linear calibration, the prior of W. G. Hunter and W. F. Lamboy (1981, Technometrics23, 323–350) is a second order probability matching prior.     

On the construction of Wold decomposition for multivariate stationary processes

A method to construct the Wold decomposition for multivariate stationary stochastic processes x_k, k Z, is presented. The method is based on orthogonal decompositions for x_k, k Z, obtained by forming orthogonal projections of x_k, k Z, onto its component processes , k Z, j = 1, …, q. The method does not give a complete solution to the Wold decomposition problem.     

Adaptive nonparametric estimation of a multivariate regression function

We consider the kernel estimation of a multivariate regression function at a point. Theoretical choices of the bandwidth are possible for attaining minimum mean squared error or for local scaling, in the sense of asymptotic distribution. However, these choices are not available in practice. We follow the approach of Krieger and Pickands (Ann. Statist.9 (1981) 1066–1078) and Abramson (J. Multivariate Anal.12 (1982), 562–567) in constructing adaptive estimates after demonstrating the weak convergence of some error process. As consequences, efficient data-driven consistent estimation is feasible, and data-driven local scaling is also feasible. In the latter instance, nearest-neighbor-type estimates and variance-stabilizing estimates are obtained as special cases.     

Continuous Algorithms in n-Term Approximation and Non-Linear Widths

In the present paper we investigate optimal continuous algorithms in n-term approximation based on various non-linear n-widths, and n-term approximation by the dictionary V formed from the integer translates of the mixed dyadic scales of the tensor product multivariate de la Vallée Poussin kernel, for the unit ball of Sobolev and Besov spaces of functions with common mixed smoothness. The asymptotic orders of these quantities are given. For each space the asymptotic orders of non-linear n-widths and n-term approximation coincide. Moreover, these asymptotic orders are achieved by a continuous algorithm of n-term approximation by V, which is explicitly constructed.     

Characterizations of multivariate normality II. Through linear regressions

It is established that a vector (X′₁, X′₂, …, X′_k) has a multivariate normal distribution if (i) for each X_i the regression on the rest is linear, (ii) the conditional distribution of X₁ about the regression does not depend on the rest of the variables, and (iii) the conditional distribution of X₂ about the regression does not depend on the rest of the variables, provided that the regression coefficients satisfy some more conditions that those given by [4]J. Multivar. Anal. 6 81–94].     

Some new approaches to multivariate probability distributions

We extend and generalize to the multivariate set-up our earlier investigations related to expected remaining life functions and general hazard measures including representations and stability theorems for arbitrary probability distributions in terms of these concepts. (The univariate case is discussed in detail in Kotz and Shanbhag, Advan. Appl. Probab. 12 (1980), 903–921.)     

Bootstrapping Autoregressive and Moving Average Parameter Estimates of Infinite Order Vector Autoregressive Processes

We consider anr-dimensional multivariate time series {y_t, tZ} which is generated by an infinite order vector autoregressive process. We show that a bootstrap procedure which works by generating time series replicates via an estimated finitek-order vector autoregressive process (k→∞ at an appropriate rate with the sample size) gives asymptotically valid approximations to the joint distribution of the growing set of estimated autoregressive coefficients and to the corresponding set of estimated moving average coefficients (impuls responses).     

On the Asymptotics of Trimmed Best k-Nets

Trimmed best k-nets were introduced in J. A. Cuesta-Albertos, A. Gordaliza and C. Matrán (1998, Statist. Probab. Lett.36, 401–413) as a robustified L_∞-based quantization procedure. This paper focuses on the asymptotics of this procedure. Also, some possible applications are briefly sketched to motivate the interest of this technique. Consistency and weak limit law are obtained in the multivariate setting. Consistency holds for absolutely continuous distributions without the (artificial) requirement of a trimming level varying with the sample size as in J. A. Cuesta-Albertos, A. Gordaliza and C. Matrán (1998, Statist. Probab. Lett.36, 401–413). The weak convergence will be stated toward a non-normal limit law at a O_P(n^−1/3) rate of convergence. An algorithm for computing trimmed best k-nets is proposed. Also a procedure is given in order to choose an appropriate number of centers, k, for a given data set.     

Zero-free regions for multivariate Tutte polynomials (alias Potts-model partition functions) of graphs and matroids

The chromatic polynomial P_G(q) of a loopless graph G is known to be non-zero (with explicitly known sign) on the intervals (−∞,0), (0,1) and (1,32/27]. Analogous theorems hold for the flow polynomial of bridgeless graphs and for the characteristic polynomial of loopless matroids. Here we exhibit all these results as special cases of more general theorems on real zero-free regions of the multivariate Tutte polynomial Z_G(q,v). The proofs are quite simple, and employ deletion–contraction together with parallel and series reduction. In particular, they shed light on the origin of the curious number 32/27.     

Higher Order Asymptotic Theory for Discriminant Analysis in Exponential Families of Distributions

This paper deals with the problem of classifying a multivariate observation X into one of two populations Π₁: p(x; w⁽¹⁾) S and Π₂: p(x; w⁽²⁾) S, where S is an exponential family of distributions and w⁽¹⁾ and w⁽²⁾ are unknown parameters. Let ; be a class of appropriate estimators ( ⁽¹⁾, ⁽²⁾) of (w⁽¹⁾, w⁽²⁾ based on training samples. Then we develop the higher order asymptotic theory for a class of classification statistics D = [ | = log{p(X; ⁽¹⁾)/p(X; ⁽²⁾)}, ( ⁽¹⁾, ⁽²⁾) ;]. The associated probabilities of misclassification of both kinds M( ) are evaluated up to second order of the reciprocal of the sample sizes. A classification statistic is said to be second order asymptotically best in D if it minimizes M( ) up to second order. A sufficient condition for to be second order asymptotically best in D is given. Our results are very general and give us a unified view in discriminant analysis. As special results, the Anderson W, the Cochran and Bliss classification statistic, and the quadratic classification statistic are shown to be second order asymptotically best in D in each suitable classification problem. Also, discriminant analysis in a curved exponential family is discussed.     

Asymptotic distributions of functions of the eigenvalues of sample covariance matrix and canonical correlation matrix in multivariate time series

Let S = (1/n) Σ_t=1ⁿ X(t) X(t)′, where X(1), …, X(n) are p × 1 random vectors with mean zero. When X(t) (t = 1, …, n) are independently and identically distributed (i.i.d.) as multivariate normal with mean vector 0 and covariance matrix Σ, many authors have investigated the asymptotic expansions for the distributions of various functions of the eigenvalues of S. In this paper, we will extend the above results to the case when {X(t)} is a Gaussian stationary process. Also we shall derive the asymptotic expansions for certain functions of the sample canonical correlations in multivariate time series. Applications of some of the results in signal processing are also discussed.     

Asymptotic Expansions for Large Deviation Probabilities of Noncentral Generalized Chi-Square Distributions

Asymptotic expansions for large deviation probabilities are used to approximate the cumulative distribution functions of noncentral generalized chi-square distributions, preferably in the far tails. The basic idea of how to deal with the tail probabilities consists in first rewriting these probabilities as large parameter values of the Laplace transform of a suitably defined function f_k; second making a series expansion of this function, and third applying a certain modification of Watson's lemma. The function f_k is deduced by applying a geometric representation formula for spherical measures to the multivariate domain of large deviations under consideration. At the so-called dominating point, the largest main curvature of the boundary of this domain tends to one as the large deviation parameter approaches infinity. Therefore, the dominating point degenerates asymptotically. For this reason the recent multivariate asymptotic expansion for large deviations in Breitung and Richter (1996, J. Multivariate Anal.58, 1–20) does not apply. Assuming a suitably parametrized expansion for the inverse g⁻¹ of the negative logarithm of the density-generating function, we derive a series expansion for the function f_k. Note that low-order coefficients from the expansion of g⁻¹ influence practically all coefficients in the expansion of the tail probabilities. As an application, classification probabilities when using the quadratic discriminant function are discussed.     

Rank procedures for some two-population multivariate extended classification problems

Given independent samples from three multivariate populations with cumulative distribution functions F⁽¹⁾(x), F⁽²⁾(x), and F⁽⁰⁾(x) = θF⁽¹⁾(x) + (1 ? θ)F⁽²⁾(x), where 0 ≤ θ ≤ 1 is unknown, the three-action problem involving decision as to whether the value of θ is high, low, or intermediate, is considered. A class of consistent procedures based on the relative spacing of three sample averages of linearly compounded rank scores is formulated. The asymptotic operating characteristics of the procedures when F⁽¹⁾ and F⁽²⁾ come close together are studied and the best choice of the compounding coefficients in terms of these considered. The consequence of using estimates of the best coefficients on the asymptotic operating characteristics is also examined.