Shift permutation invariance in linear random factor models

The objective of this paper is to consider shift invariance, a specific type of exchangeability, of random factors in linearmodels. The randomfactors are described via their covariance matrices and it is shown that shift invariance implies circular Toeplitz covariancematrices and marginally shift invariance implies block circular Toeplitz covariance matrices. In order to get interpretable linear models reparametrization is performed. It is shown that by putting restrictions on the spectrum of the shift invariant covariance matrices natural reparametrization conditions for the corresponding factors are obtained which then, among others, can be used to obtain unique parametrizations under shift invariance.      

Estimation in mixed effects model with errors in variables

In this paper, we consider a linear mixed-effects model with measurement errors in both fixed and random effects and find the moment of estimators for the parameters of interest. The strong consistency and asymptotic normality of the estimators are obtained under regularity conditions. Moreover, we obtain the strong consistent estimators of the asymptotic covariance matrices involved in the limiting theory. Simulations are reported for illustration.     

Bayesian shrinkage prediction for the regression problem

We consider Bayesian shrinkage predictions for the Normal regression problem under the frequentist Kullback-Leibler risk function.Firstly, we consider the multivariate Normal model with an unknown mean and a known covariance. While the unknown mean is fixed, the covariance of future samples can be different from that of training samples. We show that the Bayesian predictive distribution based on the uniform prior is dominated by that based on a class of priors if the prior distributions for the covariance and future covariance matrices are rotation invariant.Then, we consider a class of priors for the mean parameters depending on the future covariance matrix. With such a prior, we can construct a Bayesian predictive distribution dominating that based on the uniform prior.Lastly, applying this result to the prediction of response variables in the Normal linear regression model, we show that there exists a Bayesian predictive distribution dominating that based on the uniform prior. Minimaxity of these Bayesian predictions follows from these results.     

Bayesian modeling of several covariance matrices and some results on propriety of the posterior for linear regression with correlated and/or heterogeneous errors

We explore simultaneous modeling of several covariance matrices across groups using the spectral (eigenvalue) decomposition and modified Cholesky decomposition. We introduce several models for covariance matrices under different assumptions about the mean structure. We consider ‘dependence’ matrices, which tend to have many parameters, as constant across groups and/or parsimoniously modeled via a regression formulation. For ‘variances’, we consider both unrestricted across groups and more parsimoniously modeled via log-linear models. In all these models, we explore the propriety of the posterior when improper priors are used on the mean and ‘variance’ parameters (and in some cases, on components of the ‘dependence’ matrices). The models examined include several common Bayesian regression models, whose propriety has not been previously explored, as special cases. We propose a simple approach to weaken the assumption of constant dependence matrices in an automated fashion and describe how to compute Bayes factors to test the hypothesis of constant ‘dependence’ across groups. The models are applied to data from two longitudinal clinical studies.     

Maximum likelihood estimation of Wishart mean matrices under Löwner order restrictions

For Wishart density functions, there remains a long-time question unsolved. That is whether there exists the closed-form MLEs of mean matrices over the partially Löwner ordering sets. In this note, we provide an affirmative answer by demonstrating a unified procedure on exactly how the closed-form MLEs are obtained for the simple ordering case. Under the Kullback-Leibler loss function, a property of obtained MLEs is further studied. Some applications of the obtained closed-form MLEs, including the comparison between our ML estimates and Calvin and Dykstra's [Maximum likelihood estimation of a set of covariance matrices under Löwner order restrictions with applications to balanced multivariate variance components models, Ann. Statist. 19 (1991) 850-869.] which obtained by iterative algorithm, are also made.     

Estimation of Wishart mean matrices under simple tree ordering

For Wishart density functions, we study the risk dominance problems of the restricted maximum likelihood estimators of mean matrices with respect to the Kullback-Leibler loss function over restricted parameter space under the simple tree ordering set. The results are directly applied to the estimation of covariance matrices for the completely balanced multivariate multi-way random effects models without interactions.     

The asymptotic distribution of a goodness of fit statistic for factorial invariance

Suppose that random factor models with k factors are assumed to hold for m, p-variate populations. A model for factorial invariance has been proposed wherein the covariance or correlation matrices can be written as Σ_i = LC_iL′ + σ_i²I, where C_i is the covariance matrix of factor variables and L is a common factor loading matrix, i = 1,…, m. Also a goodness of fit statistic has been proposed for this model. The asymptotic distribution of this statistic is shown to be that of a quadratic form in normal variables. An approximation to this distribution is given and thus a test for goodness of fit is derived. The problem of dimension is considered and a numerical example is given to illustrate the results.     

Cramér-Rao bound for estimates of frequencies and damping factors of real superimposed signals with multiple poles in noise

The problem of estimating frequencies and damping factors of real superimposed signals with multiple poles in white Gaussian noise is considered. Such signals are described by real quasipolynomials, i.e. by linear combinations of real damped sinusoids multiplied by power functions. In a particular case when poles are simple, a real quasipolynomial becomes a real damped sinusoid. An explicit expression of the Cramér-Rao bound (CRB) for the estimation of frequencies and damping factors of the signals is obtained. To derive the CRB, we use the expression for the Fisher information matrix (FIM) which we obtained in a previous paper for the model of complex quasipolynomials (i.e. complex exponentials multiplied by complex polynomials). We rewrite the model of real quasipolynomials as a model of complex quasipolynomials with constraints imposed on the parameter set. Then we make use of the formula presented by Gorman and Hero that allows us to obtain the CRB for the model with constraints from the FIM for the model without constraints. The results of numerical simulations are presented and discussed.     

Relative efficiencies of estimates using patterned covariances or correlations in the multivariate normal estimation problem

Summary The relative efficiency of maximum likelihood estimates is studied when taking advantage of underlying linear patterns in the covariances or correlations when estimating covariance matrices. We compare the variances of estimates of the covariance matrix obtained under two nested patterns with the assumption that the more restricted pattern is the true state. Formulas for the asymptotic variances are given which are exact for linear covariance patterns when explicit maximum likelihood estimates exist. Several specific examples are given using complete symmetry, circular symmetry and general covariance patterns as well as an example involving a covariance matrix with a linear pattern in the correlations.     

Estimation for parameters of interest in random effects growth curve models

In this paper, we consider the general growth curve model with multivariate random effects covariance structure and provide a new simple estimator for the parameters of interest. This estimator is not only convenient for testing the hypothesis on the corresponding parameters, but also has higher efficiency than the least-square estimator and the improved two-stage estimator obtained by Rao under certain conditions. Moreover, we obtain the necessary and sufficient condition for the new estimator to be identical to the best linear unbiased estimator. Examples of its application are given.     

Predictivistic characterizations of multivariate student-t models

De Finetti style theorems characterize models (predictive distributions) as mixtures of the likelihood function and the prior distribution, beginning from some judgment of invariance about observable quantities. The likelihood function generally has its functional form identified from invariance assumptions only. However, we need additional conditions on observable quantities (typically, assumptions on conditional expectations) to identify the prior distribution. In this paper, we consider some well-known invariance assumptions and establish additional conditions on observable quantities in order to obtain a predictivistic characterization of the multivariate and matrix-variate Student-t distributions as well as for the Student-t linear model. As a byproduct, a characterization for the Pearson type II distribution is provided.     

Robust discrimination under a hierarchy on the scatter matrices

Under normality, Flury and Schmid [Quadratic discriminant functions with constraints on the covariances matrices: some asymptotic results, J. Multivariate Anal. 40 (1992) 244-261] investigated the asymptotic properties of the quadratic discrimination procedure under hierarchical models for the scatter matrices, that is: (i) arbitrary scatter matrices, (ii) common principal components, (iii) proportional scatter matrices and (iv) identical matrices. In this paper, we study the properties of robust quadratic discrimination rules based on robust estimates of the involved parameters. Our analysis is based on the partial influence functions of the functionals related to these parameters and allows to derive the asymptotic variances of the estimated coefficients under models (i)-(iv). From them, we conclude that the asymptotic variances verify the same order relations as those obtained by Flury and Schmid [Quadratic discriminant functions with constraints on the covariances matrices: some asymptotic results, J. Multivariate Anal. 40 (1992) 244-261] for the classical estimators. We also perform a Monte Carlo study for different sample sizes and different hierarchies which shows the advantage of using robust procedures over classical ones, when anomalous data are present. It also confirms that better rates of misclassification can be achieved if a more parsimonious model among all the correct ones is used instead of the standard quadratic discrimination.     

Testing some covariance structures under a growth curve model

This paper considers three types of problems: (i) the problem of independence of two sets, (ii) the problem of sphericity of the covariance matrix Σ, and (iii) the problem of intraclass model for the covariance matrix Σ, when the column vectors of X are independently distributed as multivariate normal with covariance matrix Σ and E(X) = BξA,A and B being given matrices and ξ and Σ being unknown. These problems are solved by the likelihood ratio test procedures under some restrictions on the models, and the null distributions of the test statistics are established.     

On a model selection problem from high-dimensional sample covariance matrices

Modern random matrix theory indicates that when the population size p is not negligible with respect to the sample size n, the sample covariance matrices demonstrate significant deviations from the population covariance matrices. In order to recover the characteristics of the population covariance matrices from the observed sample covariance matrices, several recent solutions are proposed when the order of the underlying population spectral distribution is known. In this paper, we deal with the underlying order selection problem and propose a solution based on the cross-validation principle. We prove the consistency of the proposed procedure.     

Semiconvexity of invariant functions of rectangular matrices

The paper deals with the semiconvexity properties (i.e., the rank 1 convexity, quasiconvexity, polyconvexity, and convexity) of rotationally invariant functions f of matrices. For the invariance with respect to the proper orthogonal group and the invariance with respect to the full orthogonal group coincide. With each invariant f one can associate a fully invariant function of a square matrix of type where It is shown that f has the semi convexity of a given type if and only if has the semiconvexity of that type. Consequently the semiconvex hulls of f can be determined by evaluating the corresponding hulls of and then extending them to matrices by rotational invariance. Received: 10 October 2001 / Accepted: 23 January 2002 // Published online: 6 August 2002 RID="*" ID="*" This research was supported by Grant 201/00/1516 of the Czech Republic.     

Similar tests for covariance structures in multivariate linear models

Nyblom (J. Multivariate Anal. 76 (2001) 294) has derived locally best invariant test for the covariance structure in a multivariate linear model. The class of invariant tests obtained by Nyblom [9] does not coincide with the class of similar tests for this testing set-up. This paper extends some of the results of Nyblom [9] by deriving the locally best similar tests for the covariance structure. Moreover, it develops a saddlepoint approximation to optimal weighted average power similar tests (i.e. tests which maximize a weighted average power).     

Estimation of means of multivariate normal populations with order restriction

Multivariate isotonic regression theory plays a key role in the field of statistical inference under order restriction for vector valued parameters. Two cases of estimating multivariate normal means under order restricted set are considered. One case is that covariance matrices are known, the other one is that covariance matrices are unknown but are restricted by partial order. This paper shows that when covariance matrices are known, the estimator given by this paper always dominates unrestricted maximum likelihood estimator uniformly, and when covariance matrices are unknown, the plug-in estimator dominates unrestricted maximum likelihood estimator under the order restricted set of covariance matrices. The isotonic regression estimators in this paper are the generalizations of plug-in estimators in unitary case.     

Positivity for Gaussian graphical models

Gaussian graphical models are parametric statistical models for jointly normal random variables whose dependence structure is determined by a graph. In previous work, we introduced trek separation, which gives a necessary and sufficient condition in terms of the graph for when a subdeterminant is zero for all covariance matrices that belong to the Gaussian graphical model. Here we extend this result to give explicit cancellation-free formulas for the expansions of non-zero subdeterminants.     

Influence analysis on linear models with random effects

In this paper,a unified diagnostic method for linear models with random effects basedupon the joint likelihood given by Robinson gin 1991) is presented. The case deletion model is e-quivalent to mean shift outlier model,as well as case weights model. From this point of view,several new diagnostic measures,such as Cook disrance,WK diagnostics are derived. Some pre-vious results are improved. Numerical examples illustrate the method is available.     

Conditional information criteria for selecting variables in linear mixed models

In this paper, we consider the problem of selecting the variables of the fixed effects in the linear mixed models where the random effects are present and the observation vectors have been obtained from many clusters. As the variable selection procedure, here we use the Akaike Information Criterion, AIC. In the context of the mixed linear models, two kinds of AIC have been proposed: marginal AIC and conditional AIC. In this paper, we derive three versions of conditional AIC depending upon different estimators of the regression coefficients and the random effects. Through the simulation studies, it is shown that the proposed conditional AIC’s are superior to the marginal and conditional AIC’s proposed in the literature in the sense of selecting the true model. Finally, the results are extended to the case when the random effects in all the clusters are of the same dimension but have a common unknown covariance matrix.