Optimal training sample allocation and asymptotic expansions for error rates in discriminant analysis

The sample-based rule obtained from Bayes classification rule by replacing the unknown parameters by ML estimates from a stratified training sample is used for the classification of a random observationX into one ofL populations. The asymptotic expansions in terms of the inverses of the training sample sizes for cross-validation, apparent and plug-in error rates are found. These are used to compare estimation methods of the error rate for a wide range of regular distributions as probability models for considered populations. The optimal training sample allocation minimizing the asymptotic expected error regret is found in the cases of widely applicable, positively skewed distributions (Rayleigh and Maxwell distributions). These probability models for populations are often met in ecology and biology. The results indicate that equal training sample sizes for each populations sometimes are not optimal, even when prior probabilities of populations are equal.     

Regularized classification for mixed continuous and categorical variables under across-location heteroscedasticity

A regularized classifier is proposed for a two-population classification problem of mixed continuous and categorical variables in a general location model(GLOM). The limiting overall expected error for the classifier is given. It can be used in an optimization search for the regularization parameters. For a heteroscedastic spherical dispersion across all locations, an asymptotic error is available which provides an alternative criterion for the optimization search. In addition, the asymptotic error can serve as a baseline for practical comparisons with other classifiers. Results based on a simulation and two real datasets are presented.     

Properties of spatial cross-periodograms using fixed-domain asymptotics

Cross-periodograms can be used to study a multivariate spatial process observed on a lattice. For spatial data, it is often appropriate to study asymptotic properties of statistical procedures under fixed-domain asymptotics in which the number of observations increases in a fixed region while shrinking distances between neighboring observations. Using fixed-domain asymptotics, we prove relative asymptotic unbiasedness and relative consistency of a smoothed cross-periodogram after appropriate filtering of the data. In addition, we show that smoothed cross-periodograms are asymptotically normal when the process is stationary multivariate Gaussian with appropriate assumptions on high-frequency behavior of the spectral density.     

Who's afraid of reduced-rank parameterizations of multivariate models? Theory and example

Reduced-rank restrictions can add useful parsimony to coefficient matrices of multivariate models, but their use is limited by the daunting complexity of the methods and their theory. The present work takes the easy road, focusing on unifying themes and simplified methods. For Gaussian and non-Gaussian (GLM, GAM, mixed normal, etc.) multivariate models, the present work gives a unified, explicit theory for the general asymptotic (normal) distribution of maximum likelihood estimators (MLE). MLE can be complex and computationally hard, but we show a strong asymptotic equivalence between MLE and a relatively simple minimum (Mahalanobis) distance estimator. The latter method yields particularly simple tests of rank, and we describe its asymptotic behavior in detail. We also examine the method's performance in simulation and via analytical and empirical examples.     

Normal theory likelihood ratio statistic for mean and covariance structure analysis under alternative hypotheses

The normal distribution based likelihood ratio (LR) statistic is widely used in structural equation modeling. Under a sequence of local alternative hypotheses, this statistic has been shown to asymptotically follow a noncentral chi-square distribution. In practice, the population mean vector and covariance matrix as well as the model and sample size are always fixed. It is hard to justify the validity of the noncentral chi-square distribution for the resulting LR statistic even when data are normally distributed and sample size is large. By extending results in the literature, this paper develops normal distributions to describe the behavior of the LR statistic for mean and covariance structure analysis. A sequence of local alternative hypotheses is not necessary for the proposed distributions to be asymptotically valid. When the effect size is medium and above or when the model is not trivially misspecified, empirical results indicate that a refined normal distribution describes the behavior of the LR statistic better than the commonly used noncentral chi-square distribution, as measured by the Kolmogorov-Smirnov distance. Quantile-quantile plots are also provided to better understand the different distributions.     

An adaptive estimation of MAVE

Minimum average variance estimation (MAVE, Xia et al. (2002) [29]) is an effective dimension reduction method. It requires no strong probabilistic assumptions on the predictors, and can consistently estimate the central mean subspace. It is applicable to a wide range of models, including time series. However, the least squares criterion used in MAVE will lose its efficiency when the error is not normally distributed. In this article, we propose an adaptive MAVE which can be adaptive to different error distributions. We show that the proposed estimate has the same convergence rate as the original MAVE. An EM algorithm is proposed to implement the new adaptive MAVE. Using both simulation studies and a real data analysis, we demonstrate the superior finite sample performance of the proposed approach over the existing least squares based MAVE when the error distribution is non-normal and the comparable performance when the error is normal.     

Nonparametric regression estimation with general parametric error covariance

The asymptotic distribution for the local linear estimator in nonparametric regression models is established under a general parametric error covariance with dependent and heterogeneously distributed regressors. A two-step estimation procedure that incorporates the parametric information in the error covariance matrix is proposed. Sufficient conditions for its asymptotic normality are given and its efficiency relative to the local linear estimator is established. We give examples of how our results are useful in some recently studied regression models. A Monte Carlo study confirms the asymptotic theory predictions and compares our estimator with some recently proposed alternative estimation procedures.     

Central limit theorems for realized volatility under hitting times of an irregular grid

We consider a continuous semi-martingale sampled at hitting times of an irregular grid. The goal of this work is to analyze the asymptotic behavior of the realized volatility under this rather natural observation scheme. This framework strongly differs from the well understood situations when the sampling times are deterministic or when the grid is regular. Indeed, neither Gaussian approximations nor symmetry properties can be used. In this setting, as the distance between two consecutive barriers tends to zero, we establish central limit theorems for the normalized error of the realized volatility. In particular, we show that there is no bias in the limiting process.     

On nonparametric classification with missing covariates

General procedures are proposed for nonparametric classification in the presence of missing covariates. Both kernel-based imputation as well as Horvitz-Thompson-type inverse weighting approaches are employed to handle the presence of missing covariates. In the case of imputation, it is a certain regression function which is being imputed (and not the missing values). Using the theory of empirical processes, the performance of the resulting classifiers is assessed by obtaining exponential bounds on the deviations of their conditional errors from that of the Bayes classifier. These bounds, in conjunction with the Borel-Cantelli lemma, immediately provide various strong consistency results.     

Selection of variables in two-group discriminant analysis by error rate and Akaike's information criteria

This paper deals with two criteria for selection of variables for the discriminant analysis in the case of two multivariate normal populations with different means and a common covariance matrix. One is based on the estimated error rate of misclassification. The other uses Akaike's information criterion. The asymptotic distributions and error rate risks of the criteria are obtained. The result will prove that the two criteria are asymptotically equivalent in the sense of their asymptotic distributions and error rate risks being identical.     

A distance-based,misclassification rate adjusted classifier for multiclass,high-dimensional data

In this paper, we consider a scale adjusted-type distance-based classifier for high-dimensional data. We first give such a classifier that can ensure high accuracy in misclassification rates for two-class classification. We show that the classifier is not only consistent but also asymptotically normal for high-dimensional data. We provide sample size determination so that misclassification rates are no more than a prespecified value. We propose a classification procedure called the misclassification rate adjusted classifier. We further develop the classifier to multiclass classification. We show that the classifier can still enjoy asymptotic properties and ensure high accuracy in misclassification rates for multiclass classification. Finally, we demonstrate the proposed classifier in actual data analyses by using a microarray data set.     

Model checking in errors-in-variables regression

This paper discusses a class of minimum distance tests for fitting a parametric regression model to a class of regression functions in the errors-in-variables model. These tests are based on certain minimized distances between a nonparametric regression function estimator and a deconvolution kernel estimator of the conditional expectation of the parametric model being fitted. The paper establishes the asymptotic normality of the proposed test statistics under the null hypothesis and that of the corresponding minimum distance estimators. We also prove the consistency of the proposed tests against a fixed alternative and obtain the asymptotic distributions for general local alternatives. Simulation studies show that the testing procedures are quite satisfactory in the preservation of the finite sample level and in terms of a power comparison.     

Some high-dimensional tests for a one-way MANOVA

A statistic is proposed for testing the equality of the mean vectors in a one-way multivariate analysis of variance. The asymptotic null distribution of this statistic, as both the sample size and the number of variables go to infinity, is shown to be normal. Thus, this test can be used when the number of variables is not small relative to the sample size. In particular, it can be used when the number of variables exceeds the degrees of freedom for error, a situation in which standard MANOVA tests are invalid. A related statistic, also having an asymptotic normal distribution, is developed for tests concerning the dimensionality of the hyperplane formed by the population mean vectors. The finite sample size performances of the normal approximations are evaluated in a simulation study.     

Minimum distance classification rules for high dimensional data

In this article, the problem of classifying a new observation vector into one of the two known groups Π_i,i=1,2, distributed as multivariate normal with common covariance matrix is considered. The total number of observation vectors from the two groups is, however, less than the dimension of the observation vectors. A sample-squared distance between the two groups, using Moore-Penrose inverse, is introduced. A classification rule based on the minimum distance is proposed to classify an observation vector into two or several groups. An expression for the error of misclassification when there are only two groups is derived for large p and n=O(p^δ),0<δ<1.     

Local linear regression for functional predictor and scalar response

The aim of this work is to introduce a new nonparametric regression technique in the context of functional covariate and scalar response. We propose a local linear regression estimator and study its asymptotic behaviour. Its finite-sample performance is compared with a Nadayara-Watson type kernel regression estimator and with the linear regression estimator via a Monte Carlo study and the analysis of two real data sets. In all the scenarios considered, the local linear regression estimator performs better than the kernel one, in the sense that the mean squared prediction error is lower.     

Rank-based inference for the single-index model

This paper is concerned with estimating the coefficients in single-index models. We develop a robust estimator, which combines the ideas of rank-based regression inference and outer product of gradients. Both asymptotic and numerical results show that the proposed procedure has better performance than the least-squares-based method when the errors deviate from normal.     

Statistical inference for panel data semiparametric partially linear regression models with heteroscedastic errors

We consider a panel data semiparametric partially linear regression model with an unknown parameter vector for the linear parametric component, an unknown nonparametric function for the nonlinear component, and a one-way error component structure which allows unequal error variances (referred to as heteroscedasticity). We develop procedures to detect heteroscedasticity and one-way error component structure, and propose a weighted semiparametric least squares estimator (WSLSE) of the parametric component in the presence of heteroscedasticity and/or one-way error component structure. This WSLSE is asymptotically more efficient than the usual semiparametric least squares estimator considered in the literature. The asymptotic properties of the WSLSE are derived. The nonparametric component of the model is estimated by the local polynomial method. Some simulations are conducted to demonstrate the finite sample performances of the proposed testing and estimation procedures. An example of application on a set of panel data of medical expenditures in Australia is also illustrated.     

Asymptotics of Bayesian median loss estimation

We establish the consistency, asymptotic normality, and efficiency for estimators derived by minimizing the median of a loss function in a Bayesian context. We contrast this procedure with the behavior of two Frequentist procedures, the least median of squares (LMS) and the least trimmed squares (LTS) estimators, in regression problems. The LMS estimator is the Frequentist version of our estimator, and the LTS estimator approaches a median-based estimator as the trimming approaches 50% on each side. We argue that the Bayesian median-based method is a good tradeoff between the two Frequentist estimators.     

Asymptotic robustness of standard errors in multilevel structural equation models

Data in social and behavioral sciences are often hierarchically organized. Multilevel statistical methodology was developed to analyze such data. Most of the procedures for analyzing multilevel data are derived from maximum likelihood based on the normal distribution assumption. Standard errors for parameter estimates in these procedures are obtained from the corresponding information matrix. Because practical data typically contain heterogeneous marginal skewnesses and kurtoses, this paper studies how nonnormally distributed data affect the standard errors of parameter estimates in a two-level structural equation model. Specifically, we study how skewness and kurtosis in one level affect standard errors of parameter estimates within its level and outside its level. We also show that, parallel to asymptotic robustness theory in conventional factor analysis, conditions exist for asymptotic robustness of standard errors in a multilevel factor analysis model.     

Estimation of the eigenvalues of noncentrality parameter matrix in noncentral Wishart distribution

We consider the problem of estimating the eigenvalues of noncentrality parameter matrix in noncentral Wishart distribution when the scale parameter is known. A decision theoretic approach is taken with squared error as the loss function. We propose two new estimators and show their superior performance to an usual estimator theoretically and numerically.