Finite-sample inference with monotone incomplete multivariate normal data, II

We continue our recent work on inference with two-step, monotone incomplete data from a multivariate normal population with mean and covariance matrix . Under the assumption that is block-diagonal when partitioned according to the two-step pattern, we derive the distributions of the diagonal blocks of and of the estimated regression matrix, . We represent in terms of independent matrices; derive its exact distribution, thereby generalizing the Wishart distribution to the setting of monotone incomplete data; and obtain saddlepoint approximations for the distributions of and its partial Iwasawa coordinates. We prove the unbiasedness of a modified likelihood ratio criterion for testing , where is a given matrix, and obtain the null and non-null distributions of the test statistic. In testing , where and are given, we prove that the likelihood ratio criterion is unbiased and obtain its null and non-null distributions. For the sphericity test, , we obtain the null distribution of the likelihood ratio criterion. In testing we show that a modified locally most powerful invariant statistic has the same distribution as a Bartlett-Pillai-Nanda trace statistic in multivariate analysis of variance.     

The Stein phenomenon for monotone incomplete multivariate normal data

We establish the Stein phenomenon in the context of two-step, monotone incomplete data drawn from , a (p+q)-dimensional multivariate normal population with mean and covariance matrix . On the basis of data consisting of n observations on all p+q characteristics and an additional N−n observations on the last q characteristics, where all observations are mutually independent, denote by the maximum likelihood estimator of . We establish criteria which imply that shrinkage estimators of James-Stein type have lower risk than under Euclidean quadratic loss. Further, we show that the corresponding positive-part estimators have lower risk than their unrestricted counterparts, thereby rendering the latter estimators inadmissible. We derive results for the case in which is block-diagonal, the loss function is quadratic and non-spherical, and the shrinkage estimator is constructed by means of a nondecreasing, differentiable function of a quadratic form in . For the problem of shrinking to a vector whose components have a common value constructed from the data, we derive improved shrinkage estimators and again determine conditions under which the positive-part analogs have lower risk than their unrestricted counterparts.     

Quadratic prediction problems in multivariate linear models

Linear and quadratic prediction problems in finite populations have become of great interest to many authors recently. In the present paper, we mainly aim to extend the problem of quadratic prediction from a general linear model, of form , to a multivariate linear model, denoted by with . Firstly, the optimal invariant quadratic unbiased (OIQU) predictor and the optimal invariant quadratic (potentially) biased (OIQB) predictor of for any particular symmetric nonnegative definite matrix satisfying are derived. Secondly, we consider predicting and . The corresponding restricted OIQU predictor and restricted OIQB predictor for them are given. In addition, we also offer four concluding remarks. One concerns the generalization of predicting and , and the others are concerned with three possible extensions from multivariate linear models to growth curve models, to restricted multivariate linear models, and to matrix elliptical linear models.     

Nonparametric comparison of regression functions

In this work, we provide a new methodology for comparing regression functions m₁ and m₂ from two samples. Since apart from smoothness no other (parametric) assumptions are required, our approach is based on a comparison of nonparametric estimators and of m₁ and m₂, respectively. The test statistics incorporate weighted differences of and computed at selected points. Since the design variables may come from different distributions, a crucial question is where to compare the two estimators. As our main results we obtain the limit distribution of (properly standardized) under the null hypothesis H₀:m₁=m₂ and under local and global alternatives. We are also able to choose the weight function so as to maximize the power. Furthermore, the tests are asymptotically distribution free under H₀ and both shift and scale invariant. Several such ’s may then be combined to get Maximin tests when the dimension of the local alternative is finite. In a simulation study we found out that our tests achieve the nominal level and already have excellent power for small to moderate sample sizes.     

On an additive decomposition of the BLUE in a multiple-partitioned linear model

Necessary and sufficient conditions are derived for the BLUE in a general multiple-partitioned linear model to be the sum of the BLUEs under the k small models , …, . Some consequences and further research topics are also given.     

Some equalities for estimations of variance components in a general linear model and its restricted and transformed models

For the unknown positive parameter σ² in a general linear model , the two commonly used estimations are the simple estimator (SE) and the minimum norm quadratic unbiased estimator (MINQUE). In this paper, we derive necessary and sufficient conditions for the equivalence of the SEs and MINQUEs of the variance component σ² in the original model ?, the restricted model , the transformed model , and the misspecified model .     

On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification

Let be identically distributed random vectors in R^d, independently drawn according to some probability density. An observation is said to be a layered nearest neighbour (LNN) of a point if the hyperrectangle defined by and contains no other data points. We first establish consistency results on , the number of LNN of . Then, given a sample of independent identically distributed random vectors from R^d×R, one may estimate the regression function by the LNN estimate , defined as an average over the Y_i’s corresponding to those which are LNN of . Under mild conditions on r, we establish the consistency of towards 0 as n→∞, for almost all and all p≥1, and discuss the links between r_n and the random forest estimates of Breiman (2001) [8]. We finally show the universal consistency of the bagged (bootstrap-aggregated) nearest neighbour method for regression and classification.     

Properties of cyclic subspace regression

Various properties of the regression vector produced by cyclic subspace regression with regard to the meancentered linear regression equation are put forth. In particular, the subspace associated with the creation of is shown to contain a basis that maximizes certain covariances with respect to , the orthogonal projection of onto a specific subspace of the range of X. This basis is constructed. Moreover, this paper shows how the maximum covariance values effect the . Several alternative representations of are also developed. These representations show that is a modified version of the l-factor principal components regression vector , with the modification occurring by a nonorthogonal projection. Additionally, these representations enable prediction properties associated with to be explicitly identified. Finally, methods for choosing factors are spelled out.     

Estimating the parametric component of nonlinear partial spline model

Consider a nonlinear partial spline model . This article studies the estimation problem of when g₀ is approximated by some graduating function. Some asymptotic results for are derived. In particular, it is shown that can be estimated with the usual parametric convergence rate without undersmoothing g₀.     

Single-index quantile regression

Nonparametric quantile regression with multivariate covariates is a difficult estimation problem due to the “curse of dimensionality”. To reduce the dimensionality while still retaining the flexibility of a nonparametric model, we propose modeling the conditional quantile by a single-index function , where a univariate link function g₀(⋅) is applied to a linear combination of covariates , often called the single-index. We introduce a practical algorithm where the unknown link function g₀(⋅) is estimated by local linear quantile regression and the parametric index is estimated through linear quantile regression. Large sample properties of estimators are studied, which facilitate further inference. Both the modeling and estimation approaches are demonstrated by simulation studies and real data applications.     

Sensitivity of GLS estimators in random effects models

This paper studies the sensitivity of random effects estimators in the one-way error component regression model. Maddala and Mount (1973) [6] give simulation evidence that in random effects models the properties of the feasible GLS estimator are not affected by the choice of the first-step estimator used for the covariance matrix. Taylor (1980) [8] gives a theoretical example of this effect. This paper provides a reason for this in terms of sensitivity. The properties of are transferred via an uncorrelated (and independent under normality) link, called sensitivity. The sensitivity statistic counteracts the improvement in . A Monte Carlo experiment illustrates the theoretical findings.     

Uniformly minimum variance nonnegative quadratic unbiased estimation in a generalized growth curve model

Consider the generalized growth curve model subject to R(X_m)⊆?⊆R(X₁), where B_i are the matrices of unknown regression coefficients, and E=(ε₁,…,ε_s)^′ and are independent and identically distributed with the same first four moments as a random vector normally distributed with mean zero and covariance matrix Σ. We derive the necessary and sufficient conditions under which the uniformly minimum variance nonnegative quadratic unbiased estimator (UMVNNQUE) of the parametric function with C≥0 exists. The necessary and sufficient conditions for a nonnegative quadratic unbiased estimator with of to be the UMVNNQUE are obtained as well.     

Strong convergence of the empirical distribution of eigenvalues of sample covariance matrices with a perturbation matrix

Let , where is a random symmetric matrix, a random symmetric matrix, and with being independent real random variables. Suppose that , and are independent. It is proved that the empirical spectral distribution of the eigenvalues of random symmetric matrices converges almost surely to a non-random distribution.     

Second order optimality for estimators in time series regression models

We consider the second order asymptotic properties of an efficient frequency domain regression coefficient estimator proposed by Hannan [Regression for time series, Proc. Sympos. Time Series Analysis (Brown Univ., 1962), Wiley, New York, 1963, pp. 17-37]. This estimator is a semiparametric estimator based on nonparametric spectral estimators. We derive the second order Edgeworth expansion of the distribution of . Then it is shown that the second order asymptotic properties are independent of the bandwidth choice for residual spectral estimator, which implies that has the same rate of convergence as in regular parametric estimation. This is a sharp contrast with the general semiparametric estimation theory. We also examine the second order Gaussian efficiency of . Numerical studies are given to confirm the theoretical results.     

Asymptotic distributions of maxima of complete and incomplete samples from multivariate stationary Gaussian sequences

Let be a sequence of d-dimensional stationary Gaussian vectors, and let denote the partial maxima of . Suppose that there are missing data in each component of and let denote the partial maxima of the observed variables. In this note, we study two kinds of asymptotic distributions of the random vector where the correlation and cross-correlation satisfy some dependence conditions.     

Density estimation by the penalized combinatorial method

Let f be an unknown multivariate density belonging to a prespecified parametric class of densities, , where k is unknown, but for all k and each has finite Vapnik-Chervonenkis dimension. Given an i.i.d. sample of size n drawn from f, we show that it is possible to select automatically, and without extra restrictions on f, an estimate with the property that . Our method is inspired by the combinatorial tools developed in Devroye and Lugosi (Combinatorial Methods in Density Estimation, Springer, New York, 2001) and it includes a wide range of density models, such as mixture models or exponential families.     

Statistical inference of minimum BD estimators and classifiers for varying-dimensional models

Stochastic modeling for large-scale datasets usually involves a varying-dimensional model space. This paper investigates the asymptotic properties, when the number of parameters grows with the available sample size, of the minimum- estimators and classifiers under a broad and important class of Bregman divergence (), which encompasses nearly all of the commonly used loss functions in the regression analysis, classification procedures and machine learning literature. Unlike the maximum likelihood estimators which require the joint likelihood of observations, the minimum-BD estimators are useful for a range of models where the joint likelihood is unavailable or incomplete. Statistical inference tools developed for the class of large dimensional minimum- estimators and related classifiers are evaluated via simulation studies, and are illustrated by analysis of a real dataset.