Maximum likelihood estimation of a multivariate linear functional relationship

Many authors have discussed maximum likelihood estimation in the simple linear functional relationship model. In this paper, we derive maximum likelihood estimators (MLEs) for parameters in a much more general model. Several special cases including the multivariate linear functional relationship model are discussed. Estimators of some of the parameters are shown to be inconsistent.     

Third-order power comparisons for a class of tests for multivariate linear hypothesis under general distributions

The purpose of this paper is, in multivariate linear regression model (Part I) and GMANOVA model (Part II), to investigate the effect of nonnormality upon the nonnull distributions of some multivariate test statistics under normality. It is shown that whatever the underlying distributions, the difference of local powers up to order N⁻¹ after either Bartlett’s type adjustment or Cornish-Fisher’s type size adjustment under nonnormality coincides with that in Anderson [An Introduction to Multivariate Statistical Analysis, 2nd ed. and 3rd ed., Wiley, New York, 1984, 2003] under normality. The derivation of asymptotic expansions is based on the differential operator associated with the multivariate linear regression model under general distributions. The performance of higher-order results in finite samples, including monotone Bartlett’s type adjustment and monotone Cornish-Fisher’s type size adjustment, is examined using simulation studies.     

Nonparametric estimation of mixed partial derivatives of a multivariate density

On the basis of a random sample of size n on an m-dimensional random vector X, this note proposes a class of estimators f_n^(p) of f^(p), where f is a density of X w.r.t. a σ-finite measure dominated by the Lebesgue measure on R^m, p = (p₁,…,p_m), p_j ≥ 0, fixed integers, and for x = (x₁,…,x_m) in R^m, f^(p)(x) = ?^p₁+…+p_m f(x)/(?^p₁x₁ … ?^p_mx_m). Asymptotic unbiasedness as well as both almost sure and mean square consistencies of f_n^(p) are examined. Further, a necessary and sufficient condition for uniform asymptotic unbisedness or for uniform mean square consistency of f_n^(p) is given. Finally, applications of estimators of this note to certain statistical problems are pointed out.     

Minimax estimation for singular linear multivariate models with mixed uncertainty

The problem of minimax estimation is examined for the linear multivariate statistically indeterminate observation model with mixed uncertainty. The a priori information on the distributions of model parameters is formulated in terms of second-order moment characteristics. It is shown that in the regular case the minimax estimate is defined explicitly via the solution of the dual optimization problem. For singular models, the method of dual optimization is developed by means of using the Tikhonov regularization techniques. Several particular cases which are widely used in practice are also considered.     

On the Maximum Likelihood estimation of a linear structural relationship when the intercept is known

This paper considers the Maximum Likelihood (ML) estimation of the five parameters of a linear structural relationship y = α + βx when α is known. The parameters are β, the two variances of observation errors on x and y, the mean and variance of x. When the ML estimates of the parameters cannot be obtained by solving a simple simultaneous system of five equations, they are found by maximizing the likelihood function directly. Some asymptotic properties of the estimates are also obtained.     

Restricted estimation in multivariate measurement error regression model

We study a multivariate ultrastructural measurement error (MUME) model with more than one response variable. This model is a synthesis of multivariate functional and structural models. Three consistent estimators of regression coefficients, satisfying the exact linear restrictions have been proposed. Their asymptotic distributions are derived under the assumption of a non-normal measurement error and random error components. A simulation study is carried out to investigate the small sample properties of the estimators. The effect of departure from normality of the measurement errors on the estimators is assessed.     

A family of estimators for multivariate kurtosis in a nonnormal linear regression model

In this paper, we propose a new estimator for a kurtosis in a multivariate nonnormal linear regression model. Usually, an estimator is constructed from an arithmetic mean of the second power of the squared sample Mahalanobis distances between observations and their estimated values. The estimator gives an underestimation and has a large bias, even if the sample size is not small. We replace this squared distance with a transformed squared norm of the Studentized residual using a monotonic increasing function. Our proposed estimator is defined by an arithmetic mean of the second power of these squared transformed squared norms with a correction term and a tuning parameter. The correction term adjusts our estimator to an unbiased estimator under normality, and the tuning parameter controls the sizes of the squared norms of the residuals. The family of our estimators includes estimators based on ordinary least squares and predicted residuals. We verify that the bias of our new estimator is smaller than usual by constructing numerical experiments.     

On the singularity of multivariate skew-symmetric models

In recent years, the skew-normal models introduced by Azzalini (1985) [1]-and their multivariate generalizations from Azzalini and Dalla Valle (1996) [4]-have enjoyed an amazing success, although an important literature has reported that they exhibit, in the vicinity of symmetry, singular Fisher information matrices and stationary points in the profile log-likelihood function for skewness, with the usual unpleasant consequences for inference. It has been shown (DiCiccio and Monti (2004) [23], DiCiccio and Monti (2009) [24] and Gómez et al. (2007) [25]) that these singularities, in some specific parametric extensions of skew-normal models (such as the classes of skew-t or skew-exponential power distributions), appear at skew-normal distributions only. Yet, an important question remains open: in broader semiparametric models of skewed distributions (such as the general skew-symmetric and skew-elliptical ones), which symmetric kernels lead to such singularities? The present paper provides an answer to this question. In very general (possibly multivariate) skew-symmetric models, we characterize, for each possible value of the rank of Fisher information matrices, the class of symmetric kernels achieving the corresponding rank. Our results show that, for strictly multivariate skew-symmetric models, not only Gaussian kernels yield singular Fisher information matrices. In contrast, we prove that systematic stationary points in the profile log-likelihood functions are obtained for (multi)normal kernels only. Finally, we also discuss the implications of such singularities on inference.     

Conditional and unconditional methods for selecting variables in linear mixed models

In the problem of selecting the explanatory variables in the linear mixed model, we address the derivation of the (unconditional or marginal) Akaike information criterion (AIC) and the conditional AIC (cAIC). The covariance matrices of the random effects and the error terms include unknown parameters like variance components, and the selection procedures proposed in the literature are limited to the cases where the parameters are known or partly unknown. In this paper, AIC and cAIC are extended to the situation where the parameters are completely unknown and they are estimated by the general consistent estimators including the maximum likelihood (ML), the restricted maximum likelihood (REML) and other unbiased estimators. We derive, related to AIC and cAIC, the marginal and the conditional prediction error criteria which select superior models in light of minimizing the prediction errors relative to quadratic loss functions. Finally, numerical performances of the proposed selection procedures are investigated through simulation studies.     

Minimax least squares and quasiminimax estimation in linear models with missing values

We consider the problem of estimating the parameter vector in the linear model when observations on the independent variables are partially missing or incorrect. New estimators are developed, which systematically combine prior information with the incomplete data. We compare these methods with the alternative strategy of deleting incomplete observations.Support by Deutsche Forschungsgemeinschaft, Grant No. 284/1-2 is gratefully acknowledged.     

Bayes estimation in linear models: A coordinate-free approach

The unified theory of Bayes estimation in linear models is presented, using a coordinate-free approach. The results are applied to the problem of linear and quadratic estimation in linear regression model.     

Structural decompositions of multivariate distributions with applications in moment and cumulant

We provide lattice decompositions for multivariate distributions. The lattice decompositions reveal the structural relationship between the Lancaster/Bahadur model and the model of Streitberg (Ann. Statist. 18 (1990) 1878). For multivariate categorical data, the decompositions allows modeling strategy for marginal inference. The theory discussed in this paper illustrates the concept of reproducibility, which was discussed in Liang et al. (J. Roy. Statist. Soc. Ser. B 54 (1992) 3). For the purpose of delineating the relationship between the various types of decompositions of distributions, we develop a theory of polytypefication, the generality of which is exploited to prove results beyond interaction.     

Estimates of MM type for the multivariate linear model

We propose a class of robust estimates for multivariate linear models. Based on the approach of MM-estimation (Yohai 1987, [24]), we estimate the regression coefficients and the covariance matrix of the errors simultaneously. These estimates have both a high breakdown point and high asymptotic efficiency under Gaussian errors. We prove consistency and asymptotic normality assuming errors with an elliptical distribution. We describe an iterative algorithm for the numerical calculation of these estimates. The advantages of the proposed estimates over their competitors are demonstrated through both simulated and real data.     

Linear plus quadratic (LPQ) quasiminimax estimation in the linear regression model

If the errors in the linear regression model are assumed to be independent with nonvanishing third and finite fourth moments, then it is possible to improve all linear estimators by so-called linear plus quadratic (LPQ) estimators. These consist of linear and quadratic terms in the endogeneous variable and depend on the unknown moments of the errors which, in general, have to be estimated from the data. In this paper, we will use LPQ estimators for quasiminimax estimation and some related problems.Support by Deutsche Forschungsgemeinschaft Grant No. Tr 253/1-2 is gratefully acknowledged.     

The asymptotic equivalence of Bayes and maximum likelihood estimation

Let (X, $\text{A}$) be a measurable space, Θ ? $\text{R}$ an open interval and P_Ω ∥ $\text{A}$, Ω ? Θ, a family of probability measures fulfilling certain regularity conditions. Let $\text{Ω}{}_{\mathrm{n}}$ be the maximum likelihood estimate for the sample size n. Let λ be a prior distribution on Θ and let $\text{R}{}_{\mathrm{<mtext>n,</mtext><mtext>x</mtext>}}$ be the posterior distribution for the sample size n given $\text{x}\text{? X}{}^{\mathrm{n}}$. $\text{L:}\text{Θ}\text{× Θ →}\text{R}$ denotes a loss function fulfilling certain regularity conditions and T_n denotes the Bayes estimate relative to λ and L for the sample size n. It is proved that for every compact K ? Θ there exists c_K ≥ 0 such that

This theorem improves results of Bickel and Yahav [3], and Ibragimov and Has'minskii [4], as far as the speed of convergence is concerned.     

A novel trace test for the mean parameters in a multivariate growth curve model

A trace test for the mean parameters of the growth curve model is proposed. It is constructed using the restricted maximum likelihood followed by an estimated likelihood ratio approach. The statistic reduces to the Lawley-Hotelling trace test for the Multivariate Analysis of Variance (MANOVA) models. Our test statistic is, therefore, a natural extension of the classical trace test to GMANOVA models. We show that the distribution of the test under the null hypothesis does not depend on the unknown covariance matrix Σ. We also show that the distributions under the null and alternative hypotheses can be represented as sums of weighted central and non-central chi-square random variables, respectively. Under the null hypothesis, the Satterthwaite approximation is used to get an approximate critical point. A novel Satterthwaite type approximation is proposed to obtain an approximate power. A simulation study is performed to evaluate the performance of our proposed test and numerical examples are provided as illustrations.     

Conditional independence models for seemingly unrelated regressions with incomplete data

We consider normal ≡ Gaussian seemingly unrelated regressions (SUR) with incomplete data (ID). Imposing a natural minimal set of conditional independence constraints, we find a restricted SUR/ID model whose likelihood function and parameter space factor into the product of the likelihood functions and the parameter spaces of standard complete data multivariate analysis of variance models. Hence, the restricted model has a unimodal likelihood and permits explicit likelihood inference. In the development of our methodology, we review and extend existing results for complete data SUR models and the multivariate ID problem.     

Relative efficiencies of estimates using patterned covariances or correlations in the multivariate normal estimation problem

Summary The relative efficiency of maximum likelihood estimates is studied when taking advantage of underlying linear patterns in the covariances or correlations when estimating covariance matrices. We compare the variances of estimates of the covariance matrix obtained under two nested patterns with the assumption that the more restricted pattern is the true state. Formulas for the asymptotic variances are given which are exact for linear covariance patterns when explicit maximum likelihood estimates exist. Several specific examples are given using complete symmetry, circular symmetry and general covariance patterns as well as an example involving a covariance matrix with a linear pattern in the correlations.     

Nonparametric methods for unbalanced multivariate data and many factor levels

We propose different nonparametric tests for multivariate data and derive their asymptotic distribution for unbalanced designs in which the number of factor levels tends to infinity (large a, small n_i case). Quasi gratis, some new parametric multivariate tests suitable for the large a asymptotic case are also obtained. Finite sample performances are investigated and compared in a simulation study. The nonparametric tests are based on separate rankings for the different variables. In the presence of outliers, the proposed nonparametric methods have better power than their parametric counterparts. Application of the new tests is demonstrated using data from plant pathology.     

Nonnegative quadratic estimation of the mean squared errors of minimax estimators in the linear regression model

In this paper, the problem of nonnegative quadratic estimation of the mean squared errors of minimax estimators of in the linear regression modelE(y)=X, VAR(y) = ² is discussed. An explicit formula for the admissible nonnegative minimum biased estimator is given. Some applications to one-way classification model are also considered.