Minimax least squares and quasiminimax estimation in linear models with missing values

We consider the problem of estimating the parameter vector in the linear model when observations on the independent variables are partially missing or incorrect. New estimators are developed, which systematically combine prior information with the incomplete data. We compare these methods with the alternative strategy of deleting incomplete observations.Support by Deutsche Forschungsgemeinschaft, Grant No. 284/1-2 is gratefully acknowledged.     

Fisher information for generalised linear mixed models

The Fisher information for the canonical link exponential family generalised linear mixed model is derived. The contribution from the fixed effects parameters is shown to have a particularly simple form.     

Conditional information criteria for selecting variables in linear mixed models

In this paper, we consider the problem of selecting the variables of the fixed effects in the linear mixed models where the random effects are present and the observation vectors have been obtained from many clusters. As the variable selection procedure, here we use the Akaike Information Criterion, AIC. In the context of the mixed linear models, two kinds of AIC have been proposed: marginal AIC and conditional AIC. In this paper, we derive three versions of conditional AIC depending upon different estimators of the regression coefficients and the random effects. Through the simulation studies, it is shown that the proposed conditional AIC’s are superior to the marginal and conditional AIC’s proposed in the literature in the sense of selecting the true model. Finally, the results are extended to the case when the random effects in all the clusters are of the same dimension but have a common unknown covariance matrix.     

Conditional and unconditional methods for selecting variables in linear mixed models

In the problem of selecting the explanatory variables in the linear mixed model, we address the derivation of the (unconditional or marginal) Akaike information criterion (AIC) and the conditional AIC (cAIC). The covariance matrices of the random effects and the error terms include unknown parameters like variance components, and the selection procedures proposed in the literature are limited to the cases where the parameters are known or partly unknown. In this paper, AIC and cAIC are extended to the situation where the parameters are completely unknown and they are estimated by the general consistent estimators including the maximum likelihood (ML), the restricted maximum likelihood (REML) and other unbiased estimators. We derive, related to AIC and cAIC, the marginal and the conditional prediction error criteria which select superior models in light of minimizing the prediction errors relative to quadratic loss functions. Finally, numerical performances of the proposed selection procedures are investigated through simulation studies.     

Minimax adjustment technique in a parameter restricted linear model

We consider an approach yielding a minimax estimator in the linear regression model with a priori information on the parameter vector, e.g., ellipsoidal restrictions. This estimator is computed directly from the loss function and can be motivated by the general Pitman nearness criterion. It turns out that this approach coincides with the projection estimator which is obtained by projecting an initial arbitrary estimate on the subset defined by the restrictions.     

Restricted estimation in multivariate measurement error regression model

We study a multivariate ultrastructural measurement error (MUME) model with more than one response variable. This model is a synthesis of multivariate functional and structural models. Three consistent estimators of regression coefficients, satisfying the exact linear restrictions have been proposed. Their asymptotic distributions are derived under the assumption of a non-normal measurement error and random error components. A simulation study is carried out to investigate the small sample properties of the estimators. The effect of departure from normality of the measurement errors on the estimators is assessed.     

The estimation of a multivariate linear relation

A multivariate linear relation η_n = β₀ξ_n is considered, in which ξ_n and η_n are observed subject to white noise errors, with covariance matrices σ₀, ω₀ respectively. If their elements lie in the null space of a suitable vector function, β₀, σ₀, ω₀ may be uniquely defined by second-order functions of the data. The asymptotic properties of estimates of β₀, σ₀, ω₀ are established under relatively mild conditions. We explore the possibility that explicit formulas for consistent estimates of β₀, σ₀, ω₀ may be available.     

A family of estimators for multivariate kurtosis in a nonnormal linear regression model

In this paper, we propose a new estimator for a kurtosis in a multivariate nonnormal linear regression model. Usually, an estimator is constructed from an arithmetic mean of the second power of the squared sample Mahalanobis distances between observations and their estimated values. The estimator gives an underestimation and has a large bias, even if the sample size is not small. We replace this squared distance with a transformed squared norm of the Studentized residual using a monotonic increasing function. Our proposed estimator is defined by an arithmetic mean of the second power of these squared transformed squared norms with a correction term and a tuning parameter. The correction term adjusts our estimator to an unbiased estimator under normality, and the tuning parameter controls the sizes of the squared norms of the residuals. The family of our estimators includes estimators based on ordinary least squares and predicted residuals. We verify that the bias of our new estimator is smaller than usual by constructing numerical experiments.     

A note on bias due to fitting prospective multivariate generalized linear models to categorical outcomes ignoring retrospective sampling schemes

Outcome-dependent sampling designs are commonly used in economics, market research and epidemiological studies. Case-control sampling design is a classic example of outcome-dependent sampling, where exposure information is collected on subjects conditional on their disease status. In many situations, the outcome under consideration may have multiple categories instead of a simple dichotomization. For example, in a case-control study, there may be disease sub-classification among the “cases” based on progression of the disease, or in terms of other histological and morphological characteristics of the disease. In this note, we investigate the issue of fitting prospective multivariate generalized linear models to such multiple-category outcome data, ignoring the retrospective nature of the sampling design. We first provide a set of necessary and sufficient conditions for the link functions that will allow for equivalence of prospective and retrospective inference for the parameters of interest. We show that for categorical outcomes, prospective-retrospective equivalence does not hold beyond the generalized multinomial logit link. We then derive an approximate expression for the bias incurred when link functions outside this class are used. Most popular models for ordinal response fall outside the multiplicative intercept class and one should be cautious while performing a naive prospective analysis of such data as the bias could be substantial. We illustrate the extent of bias through a real data example, based on the ongoing Prostate, Lung, Colorectal and Ovarian (PLCO) cancer screening trial by the National Cancer Institute. The simulations based on the real study illustrate that the bias approximations work well in practice.     

Estimates of MM type for the multivariate linear model

We propose a class of robust estimates for multivariate linear models. Based on the approach of MM-estimation (Yohai 1987, [24]), we estimate the regression coefficients and the covariance matrix of the errors simultaneously. These estimates have both a high breakdown point and high asymptotic efficiency under Gaussian errors. We prove consistency and asymptotic normality assuming errors with an elliptical distribution. We describe an iterative algorithm for the numerical calculation of these estimates. The advantages of the proposed estimates over their competitors are demonstrated through both simulated and real data.     

Nonnegative quadratic estimation and quadratic sufficiency in general linear models

Notions of linear sufficiency and quadratic sufficiency are of interest to some authors. In this paper, the problem of nonnegative quadratic estimation for β^′Hβ+hσ² is discussed in a general linear model and its transformed model. The notion of quadratic sufficiency is considered in the sense of generality, and the corresponding necessary and sufficient conditions for the transformation to be quadratically sufficient are investigated. As a direct consequence, the result on (ordinary) quadratic sufficiency is obtained. In addition, we pose a practical problem and extend a special situation to the multivariate case. Moreover, a simulated example is conducted, and applications to a model with compound symmetric covariance matrix are given. Finally, we derive a remark which indicates that our main results could be extended further to the quasi-normal case.     

M-Methods in multivariate linear models

For the multivariate linear model, coordinatewise M-estimators as well as an extension of the Maronna-type M-estimators are considered. Based on the Jure?ková (asymptotic) linearity of M-statistics, the asymptotic distribution theory of the proposed estimators is studied under appropriate regularity conditions, and incorporated in the formulation of some (asymptotic) M-tests of linear hypotheses. Finally, robustness properties of both types of estimators are discussed.     

MOMENT ESTIMATION FOR MULTIVARIATE EXTREME VALUE DISTRIBUTION

Moment estimation for multivariate extreme value distribution is described in this paper. Asymptotic covariance matrix of the estimators is given. The relative efficiencies of moment estimators as compared with the maximum likelihood and the stepwise estimators are computed. We show that when there is strong dependence between the variates, the generalized variance of moment estimators is much lower than the stepwise estimators. It becomes more obvious when the dimension increases.     

Quadratic prediction problems in multivariate linear models

Linear and quadratic prediction problems in finite populations have become of great interest to many authors recently. In the present paper, we mainly aim to extend the problem of quadratic prediction from a general linear model, of form , to a multivariate linear model, denoted by with . Firstly, the optimal invariant quadratic unbiased (OIQU) predictor and the optimal invariant quadratic (potentially) biased (OIQB) predictor of for any particular symmetric nonnegative definite matrix satisfying are derived. Secondly, we consider predicting and . The corresponding restricted OIQU predictor and restricted OIQB predictor for them are given. In addition, we also offer four concluding remarks. One concerns the generalization of predicting and , and the others are concerned with three possible extensions from multivariate linear models to growth curve models, to restricted multivariate linear models, and to matrix elliptical linear models.     

Regression models for multivariate ordered responses via the Plackett distribution

We investigate the properties of a class of discrete multivariate distributions whose univariate marginals have ordered categories, all the bivariate marginals, like in the Plackett distribution, have log-odds ratios which do not depend on cut points and all higher-order interactions are constrained to 0. We show that this class of distributions may be interpreted as a discretized version of a multivariate continuous distribution having univariate logistic marginals. Convenient features of this class relative to the class of ordered probit models (the discretized version of the multivariate normal) are highlighted. Relevant properties of this distribution like quadratic log-linear expansion, invariance to collapsing of adjacent categories, properties related to positive dependence, marginalization and conditioning are discussed briefly. When continuous explanatory variables are available, regression models may be fitted to relate the univariate logits (as in a proportional odds model) and the log-odds ratios to covariates.     

Bayesian analysis of multivariate t linear mixed models using a combination of IBF and Gibbs samplers

The multivariate linear mixed model (MLMM) has become the most widely used tool for analyzing multi-outcome longitudinal data. Although it offers great flexibility for modeling the between- and within-subject correlation among multi-outcome repeated measures, the underlying normality assumption is vulnerable to potential atypical observations. We present a fully Bayesian approach to the multivariate t linear mixed model (MtLMM), which is a robust extension of MLMM with the random effects and errors jointly distributed as a multivariate t distribution. Owing to the introduction of too many hidden variables in the model, the conventional Markov chain Monte Carlo (MCMC) method may converge painfully slowly and thus fails to provide valid inference. To alleviate this problem, a computationally efficient inverse Bayes formulas (IBF) sampler coupled with the Gibbs scheme, called the IBF-Gibbs sampler, is developed and shown to be effective in drawing samples from the target distributions. The issues related to model determination and Bayesian predictive inference for future values are also investigated. The proposed methodologies are illustrated with a real example from an AIDS clinical trial and a careful simulation study.     

Estimation in generalized linear models for functional data via penalized likelihood

We analyze in a regression setting the link between a scalar response and a functional predictor by means of a Functional Generalized Linear Model. We first give a theoretical framework and then discuss identifiability of the model. The functional coefficient of the model is estimated via penalized likelihood with spline approximation. The L² rate of convergence of this estimator is given under smoothness assumption on the functional coefficient. Heuristic arguments show how these rates may be improved for some particular frameworks.     

Estimation and inference for dependence in multivariate data

In this paper, a new measure of dependence is proposed. Our approach is based on transforming univariate data to the space where the marginal distributions are normally distributed and then, using the inverse transformation to obtain the distribution function in the original space. The pseudo-maximum likelihood method and the two-stage maximum likelihood approach are used to estimate the unknown parameters. It is shown that the estimated parameters are asymptotical normally distributed in both cases. Inference procedures for testing the independence are also studied.     

Third-order power comparisons for a class of tests for multivariate linear hypothesis under general distributions

The purpose of this paper is, in multivariate linear regression model (Part I) and GMANOVA model (Part II), to investigate the effect of nonnormality upon the nonnull distributions of some multivariate test statistics under normality. It is shown that whatever the underlying distributions, the difference of local powers up to order N⁻¹ after either Bartlett’s type adjustment or Cornish-Fisher’s type size adjustment under nonnormality coincides with that in Anderson [An Introduction to Multivariate Statistical Analysis, 2nd ed. and 3rd ed., Wiley, New York, 1984, 2003] under normality. The derivation of asymptotic expansions is based on the differential operator associated with the multivariate linear regression model under general distributions. The performance of higher-order results in finite samples, including monotone Bartlett’s type adjustment and monotone Cornish-Fisher’s type size adjustment, is examined using simulation studies.