Asymptotic expansions of the distributions of estimators in canonical correlation analysis under nonnormality

Asymptotic expansions of the distributions of typical estimators in canonical correlation analysis under nonnormality are obtained. The expansions include the Edgeworth expansions up to order O(1/n) for the parameter estimators standardized by the population standard errors, and the corresponding expansion by Hall's method with variable transformation. The expansions for the Studentized estimators are also given using the Cornish-Fisher expansion and Hall's method. The parameter estimators are dealt with in the context of estimation for the covariance structure in canonical correlation analysis. The distributions of the associated statistics (the structure of the canonical variables, the scaled log likelihood ratio and Rozeboom's between-set correlation) are also expanded. The robustness of the normal-theory asymptotic variances of the sample canonical correlations and associated statistics are shown when a latent variable model holds. Simulations are performed to see the accuracy of the asymptotic results in finite samples.     

On the estimators of model-based and maximal reliability

Four estimators of the reliability for a composite score based on the factor analysis model and five estimators of the maximal reliability for the composite are presented. When the Wishart maximum likelihood is used for the estimation of the model parameters, it is shown that the five estimators of maximal reliability are the same. Asymptotic cumulants of the estimators and their logarithmic transformations are derived under arbitrary distributions with possible model misspecification. The theoretical results considering model misspecification when a model does not hold are shown to be closer to their simulated values than those neglecting model misspecification. Simulations of the confidence intervals using the normal approximation based on the asymptotically distribution-free theory and the asymptotic expansion by Hall’s method with variable transformation are performed.     

On Some Tests of the Covariance Matrix Under General Conditions

We consider the problem of testing the hypothesis about the covariance matrix of random vectors under the assumptions that the underlying distributions are nonnormal and the sample size is moderate. The asymptotic expansions of the null distributions are obtained up to n ^−1/2. It is found that in most cases the null statistics are distributed as a mixture of independent chi-square random variables with degree of freedom one (up to n ^−1/2) and the coefficients of the mixtures are functions of the fourth cumulants of the original random variables. We also provide a general method to approximate such distributions based on a normalization transformation.     

Asymptotic expansions and higher order properties of semi-parametric estimators in a system of simultaneous equations

Asymptotic expansions are made for the distributions of the Maximum Empirical Likelihood (MEL) estimator and the Estimating Equation (EE) estimator (or the Generalized Method of Moments (GMM) in econometrics) for the coefficients of a single structural equation in a system of linear simultaneous equations, which corresponds to a reduced rank regression model. The expansions in terms of the sample size, when the non-centrality parameters increase proportionally, are carried out to O(n⁻¹). Comparisons of the distributions of the MEL and GMM estimators are made. Also, we relate the asymptotic expansions of the distributions of the MEL and GMM estimators to the corresponding expansions for the Limited Information Maximum Likelihood (LIML) and the Two-Stage Least Squares (TSLS) estimators. We give useful information on the higher order properties of alternative estimators including the semi-parametric inefficiency factor under the homoscedasticity assumption.     

Asymptotic expansions for the ability estimator in item response theory

Asymptotic approximations to the distributions of the ability estimator and its transformations in item response theory are derived beyond the usual normal one when associated item parameters are given as in tailored testing. For the approximations, the asymptotic cumulants of the estimators up to the fourth order with the higher-order asymptotic variances are obtained under possible model misspecification. For testing and interval estimation of abilities, the asymptotic cumulants of the pivots studentized in four ways are derived. Numerical examples with simulations including those for confidence intervals for abilities are given using the three-parameter logistic model.     

Nonparametric density estimation for linear processes with infinite variance

We consider nonparametric estimation of marginal density functions of linear processes by using kernel density estimators. We assume that the innovation processes are i.i.d. and have infinite-variance. We present the asymptotic distributions of the kernel density estimators with the order of bandwidths fixed as h =  cn ^−1/5, where n is the sample size. The asymptotic distributions depend on both the coefficients of linear processes and the tail behavior of the innovations. In some cases, the kernel estimators have the same asymptotic distributions as for i.i.d. observations. In other cases, the normalized kernel density estimators converge in distribution to stable distributions. A simulation study is also carried out to examine small sample properties.     

Estimation of parameters in the discrete distributions of order k

This paper considers estimating parameters in the discrete distributions of order k such as the binomial, the geometric, the Poisson and the logarithmic series distributions of order k. It is discussed how to calculate maximum likelihood estimates of parameters of the distributions based on independent observations. Further, asymptotic properties of estimators by the method of moments are investigated. In some cases, it is found that the values of asymptotic efficiency of the moment estimators are surprisingly close to one.     

Asymptotic expansions for the pivots using log-likelihood derivatives with an application in item response theory

Asymptotic expansions of the distributions of the pivotal statistics involving log-likelihood derivatives under possible model misspecification are derived using the asymptotic cumulants up to the fourth-order and the higher-order asymptotic variance. The pivots dealt with are the studentized ones by the estimated expected information, the negative Hessian matrix, the sum of products of gradient vectors, and the so-called sandwich estimator. It is shown that the first three asymptotic cumulants are the same over the pivots under correct model specification with a general condition of the equalities. An application is given in item response theory, where the observed information is usually used rather than the estimated expected one.     

Rank-Score Tests in Factorial Designs with Repeated Measures

Nonparametric factorial designs for multivariate observations are considered under the framework of general rank-score statistics. Unlike most of the literature, we do not assume the continuity of the underlying distribution functions. The models studied include general repeated measures designs, compound symmetry designs, and designs for longitudinal data. In particular, designs for ordered categorical data are included. The vectors of the multivariate observations may have different lengths. Moreover, our general framework includes missing values and singular covariance matrices which occur quite frequently in practical data analysis problems. The asymptotic properties of the proposed statistics are studied under general nonparametric hypotheses as well as under a sequence of nonparametric contiguous alternatives. L₂-consistent estimators for the unknown covariance matrices are given and two types of quadratic forms are considered for testing the nonparametric hypotheses. The results are applied to a two-way mixed model assuming compound symmetry and to a factorial design for longitudinal data. The main idea of the proofs is based on some moment inequalities for empirical distribution functions in mixed models. The details are provided in the Appendix.     

Approximations to the distribution of the sample correlation matrix

In this article, multivariate density expansions for the sample correlation matrix R are derived. The density of R is expressed through multivariate normal and through Wishart distributions. Also, an asymptotic expansion of the characteristic function of R is derived and the main terms of the first three cumulants of R are obtained in matrix form. These results make it possible to obtain asymptotic density expansions of multivariate functions of R in a direct way.     

Third-order power comparisons for a class of tests for multivariate linear hypothesis under general distributions

The purpose of this paper is, in multivariate linear regression model (Part I) and GMANOVA model (Part II), to investigate the effect of nonnormality upon the nonnull distributions of some multivariate test statistics under normality. It is shown that whatever the underlying distributions, the difference of local powers up to order N⁻¹ after either Bartlett’s type adjustment or Cornish-Fisher’s type size adjustment under nonnormality coincides with that in Anderson [An Introduction to Multivariate Statistical Analysis, 2nd ed. and 3rd ed., Wiley, New York, 1984, 2003] under normality. The derivation of asymptotic expansions is based on the differential operator associated with the multivariate linear regression model under general distributions. The performance of higher-order results in finite samples, including monotone Bartlett’s type adjustment and monotone Cornish-Fisher’s type size adjustment, is examined using simulation studies.     

On the cumulants of affine equivariant estimators in elliptical families

Given a statistical model for data which take values in R^d and have elliptically distributed errors, and affine equivariant estimators and of a mean vector in R^dRⁿ and a d × d scatter matrix, expressions are given for the covarances of the estimators in terms of their expectations and some unknown constants that depend on the model and the estimator. Higher order cumulants are also developed. These results place considerable constraints on the possible cumulants of and , as wel as those of estimators of higher order behavior such as multivariate skewness and kurtosis. These expressions are obtained using tensor methods.     

Minimum distance estimation in linear regression with unknown error distributions

This paper discusses minimum distance (m.d.) estimators of the paramter vector in the multiple linear regression model when the distributions of errors are unknown. These estimators are defined in terms of L₂-distances involving certain weighted empirical processes. Their finite sample properties and asymptotic behavior under heteroscedastic, symmetric and asymmetric errors are discussed. Some robustness properties of these estimators are also studied.     

Asymptotic properties of the Bayes modal estimators of item parameters in item response theory

Asymptotic cumulants of the Bayes modal estimators of item parameters using marginal likelihood in item response theory are derived up to the fourth order with added higher-order asymptotic variances under possible model misspecification. Among them, only the first asymptotic cumulant and the higher-order asymptotic variance for an estimator are different from those by maximum likelihood. Corresponding results for studentized Bayes estimators and asymptotically bias-corrected ones are also obtained. It was found that all the asymptotic cumulants of the bias-corrected Bayes estimator up to the fourth order and the higher-order asymptotic variance are identical to those by maximum likelihood with bias correction. Numerical illustrations are given with simulations in the case when the 2-parameter logistic model holds. In the numerical illustrations, the maximum likelihood and Bayes estimators are used, where the same independent log-normal priors are employed for discriminant parameters and the hierarchical model is adopted for the prior of difficulty parameters.     

An Asymptotic Expansion for the Distribution of Hotelling'sT-Statistic under Nonnormality

In this paper we obtain an asymptotic expansion for the distribution of Hotelling'sT²-statisticT²under nonnormality when the sample size is large. In the derivation we find an explicit Edgeworth expansion of the multivariatet-statistic. Our method is to use the Edgeworth expansion and to expand the characteristic function ofT².     

Asymptotic cumulants of the parameter estimators in item response theory

The asymptotic cumulants of the parameter estimators for the three-parameter logistic model in item response theory are derived up to the fourth order with the higher-order added asymptotic variances. The asymptotic cumulants of the corresponding Studentized estimators up to the third order are also given. The estimators are obtained by marginal maximum likelihood using the standard normal distribution for the latent variable with and without model misspecification. Numerical examples with fixed guessing parameters show advantages of the asymptotic expansions over the usual normal approximation. This work was partially supported by Grant-in-Aid for Scientific Research from the Japanese Ministry of Education, Culture, Sports, Science and Technology.     

Estimation of probability characteristics by generalized Bernstein polynomials

Approximations based on Bernstein polynomials are used for smoothing a sample quantile function and estimating the underlying distribution and its characteristics. Generalized Bernstein-type polynomials are introduced to reduce the bias of estimation under various types of distributions including finite distributions. The asymptotic behavior of the expectations of these estimators is studied. Bibliography: 10 titles.Published in Zapiski Nauchnykh Seminarov POMI, Vol. 294, 2002, pp. 127–138.     

Optimality of singular vector perturbation under maximum norm

Fan, Wang, and Zhong estimate the difference between the singular vectors of a matrix and those of a perturbed matrix in terms of the maximum norm. Their estimations are used effectively to establish the asymptotic properties of robust covariance estimators (see Journal of Machine Learning Research, 2018;18:1-42). In this paper, we give the corresponding lower bound estimates, which show Fan-Wang-Zhong's estimations optimal.     

Bootstrap method and empirical process

In this paper we consider the sampling properties of the bootstrap process, that is, the empirical process obtained from a random sample of size n (with replacement) of a fixed sample of size n of a continuous distribution. The cumulants of the bootstrap process are given up to the order n ^–1 and their unbiased estimation is discussed. Furthermore, it is shown that the bootstrap process has an asymptotic minimax property for some class of distributions up to the order n ^–1/2.     

On the limiting distribution of sample central moments

We investigate the limiting behavior of sample central moments, examining the special cases where the limiting (as the sample size tends to infinity) distribution is degenerate. Parent (non-degenerate) distributions with this property are called singular, and we show in this article that the singular distributions contain at most three supporting points. Moreover, using the delta-method, we show that the (second-order) limiting distribution of sample central moments from a singular distribution is either a multiple, or a difference of two multiples of independent Chi-square random variables with one degree of freedom. Finally, we present a new characterization of normality through the asymptotic independence of the sample mean and all sample central moments.