Conditional and unconditional methods for selecting variables in linear mixed models

In the problem of selecting the explanatory variables in the linear mixed model, we address the derivation of the (unconditional or marginal) Akaike information criterion (AIC) and the conditional AIC (cAIC). The covariance matrices of the random effects and the error terms include unknown parameters like variance components, and the selection procedures proposed in the literature are limited to the cases where the parameters are known or partly unknown. In this paper, AIC and cAIC are extended to the situation where the parameters are completely unknown and they are estimated by the general consistent estimators including the maximum likelihood (ML), the restricted maximum likelihood (REML) and other unbiased estimators. We derive, related to AIC and cAIC, the marginal and the conditional prediction error criteria which select superior models in light of minimizing the prediction errors relative to quadratic loss functions. Finally, numerical performances of the proposed selection procedures are investigated through simulation studies.     

Selecting mixed-effects models based on a generalized information criterion

The generalized information criterion (GIC) proposed by Rao and Wu [A strongly consistent procedure for model selection in a regression problem, Biometrika 76 (1989) 369-374] is a generalization of Akaike's information criterion (AIC) and the Bayesian information criterion (BIC). In this paper, we extend the GIC to select linear mixed-effects models that are widely applied in analyzing longitudinal data. The procedure for selecting fixed effects and random effects based on the extended GIC is provided. The asymptotic behavior of the extended GIC method for selecting fixed effects is studied. We prove that, under mild conditions, the selection procedure is asymptotically loss efficient regardless of the existence of a true model and consistent if a true model exists. A simulation study is carried out to empirically evaluate the performance of the extended GIC procedure. The results from the simulation show that if the signal-to-noise ratio is moderate or high, the percentages of choosing the correct fixed effects by the GIC procedure are close to one for finite samples, while the procedure performs relatively poorly when it is used to select random effects.     

On testing equality of pairwise rank correlations in a multivariate random vector

Spearman’s rank-correlation coefficient (also called Spearman’s rho) represents one of the best-known measures to quantify the degree of dependence between two random variables. As a copula-based dependence measure, it is invariant with respect to the distribution’s univariate marginal distribution functions. In this paper, we consider statistical tests for the hypothesis that all pairwise Spearman’s rank correlation coefficients in a multivariate random vector are equal. The tests are nonparametric and their asymptotic distributions are derived based on the asymptotic behavior of the empirical copula process. Only weak assumptions on the distribution function, such as continuity of the marginal distributions and continuous partial differentiability of the copula, are required for obtaining the results. A nonparametric bootstrap method is suggested for either estimating unknown parameters of the test statistics or for determining the associated critical values. We present a simulation study in order to investigate the power of the proposed tests. The results are compared to a classical parametric test for equal pairwise Pearson’s correlation coefficients in a multivariate random vector. The general setting also allows the derivation of a test for stochastic independence based on Spearman’s rho.     

REML estimation for binary data in GLMMs

The restricted maximum likelihood (REML) procedure is useful for inferences about variance components in mixed linear models. However, its extension to hierarchical generalized linear models (HGLMs) is often hampered by analytically intractable integrals. Numerical integration such as Gauss-Hermite quadrature (GHQ) is generally not recommended when the dimensionality of the integral is high. With binary data various extensions of the REML method have been suggested, but they have had unsatisfactory biases in estimation. In this paper we propose a statistically and computationally efficient REML procedure for the analysis of binary data, which is applicable over a wide class of models and design structures. We propose a bias-correction method for models such as binary matched pairs and discuss how the REML estimating equations for mixed linear models can be modified to implement more general models.     

Combining conditional and unconditional moment restrictions with missing responses

Many statistical models, e.g. regression models, can be viewed as conditional moment restrictions when distributional assumptions on the error term are not assumed. For such models, several estimators that achieve the semiparametric efficiency bound have been proposed. However, in many studies, auxiliary information is available as unconditional moment restrictions. Meanwhile, we also consider the presence of missing responses. We propose the combined empirical likelihood (CEL) estimator to incorporate such auxiliary information to improve the estimation efficiency of the conditional moment restriction models. We show that, when assuming responses are strongly ignorable missing at random, the CEL estimator achieves better efficiency than the previous estimators due to utilization of the auxiliary information. Based on the asymptotic property of the CEL estimator, we also develop Wilks’ type tests and corresponding confidence regions for the model parameter and the mean response. Since kernel smoothing is used, the CEL method may have difficulty for problems with high dimensional covariates. In such situations, we propose an instrumental variable-based empirical likelihood (IVEL) method to handle this problem. The merit of the CEL and IVEL are further illustrated through simulation studies.     

Uncertainty under a multivariate nested-error regression model with logarithmic transformation

This work aims to predict exponentials of mixed effects under a multivariate linear regression model with one random factor. Such quantities are of particular interest in prediction problems where the dependent variable is the logarithm of the variable that is the object of inference. Bias-corrected empirical predictors of the target quantities are defined. A second-order approximation for the mean crossed product error of two of these predictors is obtained, where the mean squared error is a particular case. An estimator of the mean crossed product error with second-order bias is proposed. Finally, results are illustrated through an application related to small area estimation.     

Bayesian modeling of several covariance matrices and some results on propriety of the posterior for linear regression with correlated and/or heterogeneous errors

We explore simultaneous modeling of several covariance matrices across groups using the spectral (eigenvalue) decomposition and modified Cholesky decomposition. We introduce several models for covariance matrices under different assumptions about the mean structure. We consider ‘dependence’ matrices, which tend to have many parameters, as constant across groups and/or parsimoniously modeled via a regression formulation. For ‘variances’, we consider both unrestricted across groups and more parsimoniously modeled via log-linear models. In all these models, we explore the propriety of the posterior when improper priors are used on the mean and ‘variance’ parameters (and in some cases, on components of the ‘dependence’ matrices). The models examined include several common Bayesian regression models, whose propriety has not been previously explored, as special cases. We propose a simple approach to weaken the assumption of constant dependence matrices in an automated fashion and describe how to compute Bayes factors to test the hypothesis of constant ‘dependence’ across groups. The models are applied to data from two longitudinal clinical studies.     

Marginal permutation invariant covariance matrices with applications to linear models

The goal of the present paper is to perform a comprehensive study of the covariance structures in balanced linear models containing random factors which are invariant with respect to marginal permutations of the random factors. We shall focus on model formulation and interpretation rather than the estimation of parameters. It is proven that permutation invariance implies a specific structure for the covariance matrices. Useful results are obtained for the spectra of permutation invariant covariance matrices. In particular, the reparameterization of random effects, i.e., imposing certain constraints, will be considered. There are many possibilities to choose reparameterization constraints in a linear model, however not every reparameterization keeps permutation invariance. The question is if there are natural restrictions on the random effects in a given model, i.e., such reparameterizations which are defined by the covariance structure of the corresponding factor. Examining relationships between the reparameterization conditions applied to the random factors of the models and the spectrum of the corresponding covariance matrices when permutation invariance is assumed, restrictions on the spectrum of the covariance matrix are obtained which lead to “sum-to-zero” reparameterization of the corresponding factor.     

Detection of a change point based on local-likelihood

In this paper, we consider the regression function or its νth derivative in generalized linear models which may have a change/discontinuity point at an unknown location. The location and its jump size are estimated with the local polynomial fits based on one-sided kernel weighted local-likelihood functions. Asymptotic distributions of the proposed estimators of location and jump size are established. The finite-sample performances of the proposed estimators with practical aspects are illustrated by simulated and beetle mortality examples.     

Restricted estimation in multivariate measurement error regression model

We study a multivariate ultrastructural measurement error (MUME) model with more than one response variable. This model is a synthesis of multivariate functional and structural models. Three consistent estimators of regression coefficients, satisfying the exact linear restrictions have been proposed. Their asymptotic distributions are derived under the assumption of a non-normal measurement error and random error components. A simulation study is carried out to investigate the small sample properties of the estimators. The effect of departure from normality of the measurement errors on the estimators is assessed.     

Best linear unbiased prediction for linear combinations in general mixed linear models

The general mixed linear model can be written as . In this paper, we mainly deal with two problems. Firstly, the problem of predicting a general linear combination of fixed effects and realized values of random effects in a general mixed linear model is considered and an explicit representation of the best linear unbiased predictor (BLUP) is derived. In addition, we apply the resulting conclusion to several special models and offer an alternative to characterization of BLUP. Secondly, we recall the notion of linear sufficiency and consider it as regards the BLUP problem and characterize it in several different ways. Further, we study the concepts of linear sufficiency, linear minimal sufficiency and linear completeness, and give relations among them. Finally, four concluding remarks are given.     

Linear mixed models and penalized least squares

Linear mixed-effects models are an important class of statistical models that are used directly in many fields of applications and also are used as iterative steps in fitting other types of mixed-effects models, such as generalized linear mixed models. The parameters in these models are typically estimated by maximum likelihood or restricted maximum likelihood. In general, there is no closed-form solution for these estimates and they must be determined by iterative algorithms such as EM iterations or general nonlinear optimization. Many of the intermediate calculations for such iterations have been expressed as generalized least squares problems. We show that an alternative representation as a penalized least squares problem has many advantageous computational properties including the ability to evaluate explicitly a profiled log-likelihood or log-restricted likelihood, the gradient and Hessian of this profiled objective, and an ECME update to refine this objective.     

Likelihood and conditional likelihood inference for generalized additive mixed models for clustered data

Lin and Zhang (J. Roy. Statist. Soc. Ser. B 61 (1999) 381) proposed the generalized additive mixed model (GAMM) as a framework for analysis of correlated data, where normally distributed random effects are used to account for correlation in the data, and proposed to use double penalized quasi-likelihood (DPQL) to estimate the nonparametric functions in the model and marginal likelihood to estimate the smoothing parameters and variance components simultaneously. However, the normal distributional assumption for the random effects may not be realistic in many applications, and it is unclear how violation of this assumption affects ensuing inferences for GAMMs. For a particular class of GAMMs, we propose a conditional estimation procedure built on a conditional likelihood for the response given a sufficient statistic for the random effect, treating the random effect as a nuisance parameter, which thus should be robust to its distribution. In extensive simulation studies, we assess performance of this estimator under a range of conditions and use it as a basis for comparison to DPQL to evaluate the impact of violation of the normality assumption. The procedure is illustrated with application to data from the Multicenter AIDS Cohort Study (MACS).     

Quantile regression for longitudinal data

The penalized least squares interpretation of the classical random effects estimator suggests a possible way forward for quantile regression models with a large number of “fixed effects”. The introduction of a large number of individual fixed effects can significantly inflate the variability of estimates of other covariate effects. Regularization, or shrinkage of these individual effects toward a common value can help to modify this inflation effect. A general approach to estimating quantile regression models for longitudinal data is proposed employing ?₁ regularization methods. Sparse linear algebra and interior point methods for solving large linear programs are essential computational tools.     

Conditionally specified models and dimension reduction in the exponential families

We consider informative dimension reduction for regression problems with random predictors. Based on the conditional specification of the model, we develop a methodology for replacing the predictors with a smaller number of functions of the predictors. We apply the method to the case where the inverse conditional model is in the linear exponential family. For such an inverse model and the usual Normal forward regression model it is shown that, for any number of predictors, the sufficient summary has dimension two or less. In addition, we develop a test of dimensionality. The relationship of our method with the existing dimension reduction theory based on the marginal distribution of the predictors is discussed.     

Estimation in partially linear models with missing responses at random

A partially linear model is considered when the responses are missing at random. Imputation, semiparametric regression surrogate and inverse marginal probability weighted approaches are developed to estimate the regression coefficients and the nonparametric function, respectively. All the proposed estimators for the regression coefficients are shown to be asymptotically normal, and the estimators for the nonparametric function are proved to converge at an optimal rate. A simulation study is conducted to compare the finite sample behavior of the proposed estimators.     

Empirical likelihood for the parametric part in partially linear errors-in-function models

Partially linear errors-in-function models were proposed by Liang (2000), but their inferences have not been systematically studied. This article proposes an empirical likelihood method to construct confidence regions of the parametric components. Under mild regularity conditions, the nonparametric version of the Wilk’s theorem is derived. Simulation studies show that the proposed empirical likelihood method provides narrower confidence regions, as well as higher coverage probabilities than those based on the traditional normal approximation method.     

Fisher information for generalised linear mixed models

The Fisher information for the canonical link exponential family generalised linear mixed model is derived. The contribution from the fixed effects parameters is shown to have a particularly simple form.     

Third-order power comparisons for a class of tests for multivariate linear hypothesis under general distributions

The purpose of this paper is, in multivariate linear regression model (Part I) and GMANOVA model (Part II), to investigate the effect of nonnormality upon the nonnull distributions of some multivariate test statistics under normality. It is shown that whatever the underlying distributions, the difference of local powers up to order N⁻¹ after either Bartlett’s type adjustment or Cornish-Fisher’s type size adjustment under nonnormality coincides with that in Anderson [An Introduction to Multivariate Statistical Analysis, 2nd ed. and 3rd ed., Wiley, New York, 1984, 2003] under normality. The derivation of asymptotic expansions is based on the differential operator associated with the multivariate linear regression model under general distributions. The performance of higher-order results in finite samples, including monotone Bartlett’s type adjustment and monotone Cornish-Fisher’s type size adjustment, is examined using simulation studies.     

Measures of influence for the functional linear model with scalar response

This paper studies how to identify influential observations in the functional linear model in which the predictor is functional and the response is scalar. Measurement of the effects of a single observation on estimation and prediction when the model is estimated by the principal components method is undertaken. For that, three statistics are introduced for measuring the influence of each observation on estimation and prediction of the functional linear model with scalar response that are generalizations of the measures proposed for the standard regression model by [D.R. Cook, Detection of influential observations in linear regression, Technometrics 19 (1977) 15-18; D. Peña, A new statistic for influence in linear regression, Technometrics 47 (2005) 1-12] respectively. A smoothed bootstrap method is proposed to estimate the quantiles of the influence measures, which allows us to point out which observations have the larger influence on estimation and prediction. The behavior of the three statistics and the quantile estimation bootstrap based method is analyzed via a simulation study. Finally, the practical use of the proposed statistics is illustrated by the analysis of a real data example, which show that the proposed measures are useful for detecting heterogeneity in the functional linear model with scalar response.