Existence of MLEs for discrete linear exponential models

Necessary and sufficient conditions are given for the existence of an MLE for log-linear and regression models for contingency tables. A partial compactification of the parameter space is used to elucidate a more abstract compactification given by Lauritzen [14]. A modification of the Newton-Raphson approximation yields MLEs in the partial compactification. This work was done while the author was Assistant Professor, Department of Mathematics, Arizona State University, Tempe 85281.     

Preliminary test estimators and phi-divergence measures in generalized linear models with binary data

We consider the problem of estimation of the parameters in Generalized Linear Models (GLM) with binary data when it is suspected that the parameter vector obeys some exact linear restrictions which are linearly independent with some degree of uncertainty. Based on minimum -divergence estimation (ME), we consider some estimators for the parameters of the GLM: Unrestricted ME, restricted ME, Preliminary ME, Shrinkage ME, Shrinkage preliminary ME, James–Stein ME, Positive-part of Stein-Rule ME and Modified preliminary ME. Asymptotic bias as well as risk with a quadratic loss function are studied under contiguous alternative hypotheses. Some discussion about dominance among the estimators studied is presented. Finally, a simulation study is carried out.     

General canonical correlations with applications to group symmetry models

In this paper, we define general canonical correlations, which generalize the canonical correlations developed by Hotelling, and general canonical covariate pairs, the corresponding linear statistic. We also define canonical variance distances with corresponding canonical distance variates. In a rather broad setting, these parameters and their corresponding linear statistics are characterized in terms of certain eigenvalues and eigenvectors. For seven of the ten group symmetry testing problems discussed in Andersson, Brøns, and Jensen (1983) [4], these are the eigenvalues used to represent the maximal invariant statistic, and additional observations regarding the canonical correlations are made for these testing problems.     

Strong consistency of least-squares estimation in linear regression models with vague concepts

Linear regression models with vague concepts extend the classical single equation linear regression models by admitting observations in form of fuzzy subsets instead of real numbers. They have recently been introduced [cf. Krätschmer, Induktive statistik auf basis unscharfer meßkonzepte am beispiel linearer regressionsmodelle, Unpublished Habilitation Monograph, Faculty of Law and Economics of the University of Saarland, Saarbrücken, 2001] to improve the empirical meaningfulness of the relationship between the involved items by a more sensitive attention to the problems of data measurement, in particular the fundamental problem of adequacy. The parameters of such models are still real numbers, and a method of estimation can be applied which extends directly the ordinary least-squares method. This paper deals with some first asymptotic properties of estimators obtained by the method. Firstly, strong consistency will be shown, and secondly, the convergence rate will be investigated. The later result will be the starting point for a future study which will calculate the limit distributions of the estimators.     

Confidence sets and coverage probabilities based on preliminary estimators in logistic regression models

In this paper we present recentered confidence sets for the parameters of a logistic regression model based on preliminary minimum ?$?$-divergence estimators. Asymptotic coverage probabilities are given as well as a simulation study in order to analyze the coverage probabilities for small and moderate sample sizes.     

Best linear unbiased prediction for linear combinations in general mixed linear models

The general mixed linear model can be written as . In this paper, we mainly deal with two problems. Firstly, the problem of predicting a general linear combination of fixed effects and realized values of random effects in a general mixed linear model is considered and an explicit representation of the best linear unbiased predictor (BLUP) is derived. In addition, we apply the resulting conclusion to several special models and offer an alternative to characterization of BLUP. Secondly, we recall the notion of linear sufficiency and consider it as regards the BLUP problem and characterize it in several different ways. Further, we study the concepts of linear sufficiency, linear minimal sufficiency and linear completeness, and give relations among them. Finally, four concluding remarks are given.     

Eigenvalues of large sample covariance matrices of spiked population models

We consider a spiked population model, proposed by Johnstone, in which all the population eigenvalues are one except for a few fixed eigenvalues. The question is to determine how the sample eigenvalues depend on the non-unit population ones when both sample size and population size become large. This paper completely determines the almost sure limits of the sample eigenvalues in a spiked model for a general class of samples.     

Bayesian modeling of several covariance matrices and some results on propriety of the posterior for linear regression with correlated and/or heterogeneous errors

We explore simultaneous modeling of several covariance matrices across groups using the spectral (eigenvalue) decomposition and modified Cholesky decomposition. We introduce several models for covariance matrices under different assumptions about the mean structure. We consider ‘dependence’ matrices, which tend to have many parameters, as constant across groups and/or parsimoniously modeled via a regression formulation. For ‘variances’, we consider both unrestricted across groups and more parsimoniously modeled via log-linear models. In all these models, we explore the propriety of the posterior when improper priors are used on the mean and ‘variance’ parameters (and in some cases, on components of the ‘dependence’ matrices). The models examined include several common Bayesian regression models, whose propriety has not been previously explored, as special cases. We propose a simple approach to weaken the assumption of constant dependence matrices in an automated fashion and describe how to compute Bayes factors to test the hypothesis of constant ‘dependence’ across groups. The models are applied to data from two longitudinal clinical studies.     

On the quadratic estimation of covariance matrices in multivariate linear models

This paper investigates the estimation of covariance matrices in multivariate mixed models. Some sufficient conditions are derived for a multivariate quadratic form and a linear combination of multivariate quadratic forms to be the BQUE (quadratic unbiased and severally minimum varianced) estimators of its expectations.     

On the equivalence of estimations under a general linear model and its transformed models

Some necessary and sufficient conditions are given for two equalities of ordinary least-squares estimators and best linear unbiased estimators of an estimable vector of parametric functions under a general linear model and its transformed linear model to hold     

Limit distributions of least squares estimators in linear regression models with vague concepts

Linear regression models with vague concepts extend the classical single equation linear regression models by admitting observations in form of fuzzy subsets instead of real numbers. They have lately been introduced (cf. [V. Krätschmer, Induktive Statistik auf Basis unscharfer Meßkonzepte am Beispiel linearer Regressionsmodelle, unpublished postdoctoral thesis, Faculty of Law and Economics of the University of Saarland, Saarbrücken, 2001; V. Krätschmer, Least squares estimation in linear regression models with vague concepts, Fuzzy Sets and Systems, accepted for publication]) to improve the empirical meaningfulness of the relationships between the involved items by a more sensitive attention to the problems of data measurement, in particular, the fundamental problem of adequacy. The parameters of such models are still real numbers, and a method of estimation can be applied which extends directly the ordinary least squares method. In another recent contribution (cf. [V. Krätschmer, Strong consistency of least squares estimation in linear regression models with vague concepts, J. Multivar. Anal., accepted for publication]) strong consistency and -consistency of this generalized least squares estimation have been shown. The aim of the paper is to complete these results by an investigation of the limit distributions of the estimators. It turns out that the classical results can be transferred, in some cases even asymptotic normality holds.     

Empirical likelihood for the parametric part in partially linear errors-in-function models

Partially linear errors-in-function models were proposed by Liang (2000), but their inferences have not been systematically studied. This article proposes an empirical likelihood method to construct confidence regions of the parametric components. Under mild regularity conditions, the nonparametric version of the Wilk’s theorem is derived. Simulation studies show that the proposed empirical likelihood method provides narrower confidence regions, as well as higher coverage probabilities than those based on the traditional normal approximation method.     

Estimation in partially linear models with missing responses at random

A partially linear model is considered when the responses are missing at random. Imputation, semiparametric regression surrogate and inverse marginal probability weighted approaches are developed to estimate the regression coefficients and the nonparametric function, respectively. All the proposed estimators for the regression coefficients are shown to be asymptotically normal, and the estimators for the nonparametric function are proved to converge at an optimal rate. A simulation study is conducted to compare the finite sample behavior of the proposed estimators.     

Modified Whittle estimation of multilateral models on a lattice

In the estimation of parametric models for stationary spatial or spatio-temporal data on a d-dimensional lattice, for d?2, the achievement of asymptotic efficiency under Gaussianity, and asymptotic normality more generally, with standard convergence rate, faces two obstacles. One is the “edge effect”, which worsens with increasing d. The other is the possible difficulty of computing a continuous-frequency form of Whittle estimate or a time domain Gaussian maximum likelihood estimate, due mainly to the Jacobian term. This is especially a problem in “multilateral” models, which are naturally expressed in terms of lagged values in both directions for one or more of the d dimensions. An extension of the discrete-frequency Whittle estimate from the time series literature deals conveniently with the computational problem, but when subjected to a standard device for avoiding the edge effect has disastrous asymptotic performance, along with finite sample numerical drawbacks, the objective function lacking a minimum-distance interpretation and losing any global convexity properties. We overcome these problems by first optimizing a standard, guaranteed non-negative, discrete-frequency, Whittle function, without edge-effect correction, providing an estimate with a slow convergence rate, then improving this by a sequence of computationally convenient approximate Newton iterations using a modified, almost-unbiased periodogram, the desired asymptotic properties being achieved after finitely many steps. The asymptotic regime allows increase in both directions of all d dimensions, with the central limit theorem established after re-ordering as a triangular array. However our work offers something new for “unilateral” models also. When the data are non-Gaussian, asymptotic variances of all parameter estimates may be affected, and we propose consistent, non-negative definite estimates of the asymptotic variance matrix.     

Kronecker product permutation matrices and their application to moment matrices of the normal distribution

In this paper, we consider the matrix which transforms a Kronecker product of vectors into the average of all vectors obtained by permuting the vectors involved in the Kronecker product. An explicit expression is given for this matrix, and some of its properties are derived. It is shown that this matrix is particularly useful in obtaining compact expressions for the moment matrices of the normal distribution. The utility of these expressions is illustrated through some examples.     

Sparse graphical models for exploring gene expression data

We discuss the theoretical structure and constructive methodology for large-scale graphical models, motivated by their potential in evaluating and aiding the exploration of patterns of association in gene expression data. The theoretical discussion covers basic ideas and connections between Gaussian graphical models, dependency networks and specific classes of directed acyclic graphs we refer to as compositional networks. We describe a constructive approach to generating interesting graphical models for very high-dimensional distributions that builds on the relationships between these various stylized graphical representations. Issues of consistency of models and priors across dimension are key. The resulting methods are of value in evaluating patterns of association in large-scale gene expression data with a view to generating biological insights about genes related to a known molecular pathway or set of specified genes. Some initial examples relate to the estrogen receptor pathway in breast cancer, and the Rb-E2F cell proliferation control pathway.     

On Hadamard differentiability in k-sample semiparametric models—with applications to the assessment of structural relationships

Semiparametric models to describe the functional relationship between k groups of observations are broadly applied in statistical analysis, ranging from nonparametric ANOVA to proportional hazard (ph) rate models in survival analysis. In this paper we deal with the empirical assessment of the validity of such a model, which will be denoted as a “structural relationship model”. To this end Hadamard differentiability of a suitable goodness-of-fit measure in the k-sample case is proved. This yields asymptotic limit laws which are applied to construct tests for various semiparametric models, including the Cox ph model. Two types of asymptotics are obtained, first when the hypothesis of the semiparametric model under investigation holds true, and second for the case when a fixed alternative is present. The latter result can be used to validate the presence of a semiparametric model instead of simply checking the null hypothesis “the model holds true”. Finally, various bootstrap approximations are numerically investigated and a data example is analyzed.     

An estimation method for the Neyman chi-square divergence with application to test of hypotheses

We propose a new definition of the Neyman chi-square divergence between distributions. Based on convexity properties and duality, this version of the χ² is well suited both for the classical applications of the χ² for the analysis of contingency tables and for the statistical tests in parametric models, for which it is advocated to be robust against outliers.We present two applications in testing. In the first one, we deal with goodness-of-fit tests for finite and infinite numbers of linear constraints; in the second one, we apply χ²-methodology to parametric testing against contamination.     

Fisher information for generalised linear mixed models

The Fisher information for the canonical link exponential family generalised linear mixed model is derived. The contribution from the fixed effects parameters is shown to have a particularly simple form.     

Semi-parametric estimation of partially linear single-index models

One of the most difficult problems in applications of semi-parametric partially linear single-index models (PLSIM) is the choice of pilot estimators and complexity parameters which may result in radically different estimators. Pilot estimators are often assumed to be root-n consistent, although they are not given in a constructible way. Complexity parameters, such as a smoothing bandwidth are constrained to a certain speed, which is rarely determinable in practical situations.In this paper, efficient, constructible and practicable estimators of PLSIMs are designed with applications to time series. The proposed technique answers two questions from Carroll et al. [Generalized partially linear single-index models, J. Amer. Statist. Assoc. 92 (1997) 477-489]: no root-n pilot estimator for the single-index part of the model is needed and complexity parameters can be selected at the optimal smoothing rate. The asymptotic distribution is derived and the corresponding algorithm is easily implemented. Examples from real data sets (credit-scoring and environmental statistics) illustrate the technique and the proposed methodology of minimum average variance estimation (MAVE).