Estimation in generalized linear models for functional data via penalized likelihood

We analyze in a regression setting the link between a scalar response and a functional predictor by means of a Functional Generalized Linear Model. We first give a theoretical framework and then discuss identifiability of the model. The functional coefficient of the model is estimated via penalized likelihood with spline approximation. The L² rate of convergence of this estimator is given under smoothness assumption on the functional coefficient. Heuristic arguments show how these rates may be improved for some particular frameworks.     

Confidence sets and coverage probabilities based on preliminary estimators in logistic regression models

In this paper we present recentered confidence sets for the parameters of a logistic regression model based on preliminary minimum ?$?$-divergence estimators. Asymptotic coverage probabilities are given as well as a simulation study in order to analyze the coverage probabilities for small and moderate sample sizes.     

Selecting mixed-effects models based on a generalized information criterion

The generalized information criterion (GIC) proposed by Rao and Wu [A strongly consistent procedure for model selection in a regression problem, Biometrika 76 (1989) 369-374] is a generalization of Akaike's information criterion (AIC) and the Bayesian information criterion (BIC). In this paper, we extend the GIC to select linear mixed-effects models that are widely applied in analyzing longitudinal data. The procedure for selecting fixed effects and random effects based on the extended GIC is provided. The asymptotic behavior of the extended GIC method for selecting fixed effects is studied. We prove that, under mild conditions, the selection procedure is asymptotically loss efficient regardless of the existence of a true model and consistent if a true model exists. A simulation study is carried out to empirically evaluate the performance of the extended GIC procedure. The results from the simulation show that if the signal-to-noise ratio is moderate or high, the percentages of choosing the correct fixed effects by the GIC procedure are close to one for finite samples, while the procedure performs relatively poorly when it is used to select random effects.     

Limit distributions of least squares estimators in linear regression models with vague concepts

Linear regression models with vague concepts extend the classical single equation linear regression models by admitting observations in form of fuzzy subsets instead of real numbers. They have lately been introduced (cf. [V. Krätschmer, Induktive Statistik auf Basis unscharfer Meßkonzepte am Beispiel linearer Regressionsmodelle, unpublished postdoctoral thesis, Faculty of Law and Economics of the University of Saarland, Saarbrücken, 2001; V. Krätschmer, Least squares estimation in linear regression models with vague concepts, Fuzzy Sets and Systems, accepted for publication]) to improve the empirical meaningfulness of the relationships between the involved items by a more sensitive attention to the problems of data measurement, in particular, the fundamental problem of adequacy. The parameters of such models are still real numbers, and a method of estimation can be applied which extends directly the ordinary least squares method. In another recent contribution (cf. [V. Krätschmer, Strong consistency of least squares estimation in linear regression models with vague concepts, J. Multivar. Anal., accepted for publication]) strong consistency and -consistency of this generalized least squares estimation have been shown. The aim of the paper is to complete these results by an investigation of the limit distributions of the estimators. It turns out that the classical results can be transferred, in some cases even asymptotic normality holds.     

REML estimation for binary data in GLMMs

The restricted maximum likelihood (REML) procedure is useful for inferences about variance components in mixed linear models. However, its extension to hierarchical generalized linear models (HGLMs) is often hampered by analytically intractable integrals. Numerical integration such as Gauss-Hermite quadrature (GHQ) is generally not recommended when the dimensionality of the integral is high. With binary data various extensions of the REML method have been suggested, but they have had unsatisfactory biases in estimation. In this paper we propose a statistically and computationally efficient REML procedure for the analysis of binary data, which is applicable over a wide class of models and design structures. We propose a bias-correction method for models such as binary matched pairs and discuss how the REML estimating equations for mixed linear models can be modified to implement more general models.     

Marginal permutation invariant covariance matrices with applications to linear models

The goal of the present paper is to perform a comprehensive study of the covariance structures in balanced linear models containing random factors which are invariant with respect to marginal permutations of the random factors. We shall focus on model formulation and interpretation rather than the estimation of parameters. It is proven that permutation invariance implies a specific structure for the covariance matrices. Useful results are obtained for the spectra of permutation invariant covariance matrices. In particular, the reparameterization of random effects, i.e., imposing certain constraints, will be considered. There are many possibilities to choose reparameterization constraints in a linear model, however not every reparameterization keeps permutation invariance. The question is if there are natural restrictions on the random effects in a given model, i.e., such reparameterizations which are defined by the covariance structure of the corresponding factor. Examining relationships between the reparameterization conditions applied to the random factors of the models and the spectrum of the corresponding covariance matrices when permutation invariance is assumed, restrictions on the spectrum of the covariance matrix are obtained which lead to “sum-to-zero” reparameterization of the corresponding factor.     

Fisher information for generalised linear mixed models

The Fisher information for the canonical link exponential family generalised linear mixed model is derived. The contribution from the fixed effects parameters is shown to have a particularly simple form.     

Statistical inference of minimum BD estimators and classifiers for varying-dimensional models

Stochastic modeling for large-scale datasets usually involves a varying-dimensional model space. This paper investigates the asymptotic properties, when the number of parameters grows with the available sample size, of the minimum- estimators and classifiers under a broad and important class of Bregman divergence (), which encompasses nearly all of the commonly used loss functions in the regression analysis, classification procedures and machine learning literature. Unlike the maximum likelihood estimators which require the joint likelihood of observations, the minimum-BD estimators are useful for a range of models where the joint likelihood is unavailable or incomplete. Statistical inference tools developed for the class of large dimensional minimum- estimators and related classifiers are evaluated via simulation studies, and are illustrated by analysis of a real dataset.     

Conditional information criteria for selecting variables in linear mixed models

In this paper, we consider the problem of selecting the variables of the fixed effects in the linear mixed models where the random effects are present and the observation vectors have been obtained from many clusters. As the variable selection procedure, here we use the Akaike Information Criterion, AIC. In the context of the mixed linear models, two kinds of AIC have been proposed: marginal AIC and conditional AIC. In this paper, we derive three versions of conditional AIC depending upon different estimators of the regression coefficients and the random effects. Through the simulation studies, it is shown that the proposed conditional AIC’s are superior to the marginal and conditional AIC’s proposed in the literature in the sense of selecting the true model. Finally, the results are extended to the case when the random effects in all the clusters are of the same dimension but have a common unknown covariance matrix.     

Corrected version of AIC for selecting multivariate normal linear regression models in a general nonnormal case

This paper deals with the bias reduction of Akaike information criterion (AIC) for selecting variables in multivariate normal linear regression models when the true distribution of observation is an unknown nonnormal distribution. We propose a corrected version of AIC which is partially constructed by the jackknife method and is adjusted to the exact unbiased estimator of the risk when the candidate model includes the true model. It is pointed out that the influence of nonnormality in the bias of our criterion is smaller than the ones in AIC and TIC. We verify that our criterion is better than the AIC, TIC and EIC by conducting numerical experiments.     

On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding

We study the distributions of the LASSO, SCAD, and thresholding estimators, in finite samples and in the large-sample limit. The asymptotic distributions are derived for both the case where the estimators are tuned to perform consistent model selection and for the case where the estimators are tuned to perform conservative model selection. Our findings complement those of Knight and Fu [K. Knight, W. Fu, Asymptotics for lasso-type estimators, Annals of Statistics 28 (2000) 1356–1378] and Fan and Li [J. Fan, R. Li, Variable selection via non-concave penalized likelihood and its oracle properties, Journal of the American Statistical Association 96 (2001) 1348–1360]. We show that the distributions are typically highly non-normal regardless of how the estimator is tuned, and that this property persists in large samples. The uniform convergence rate of these estimators is also obtained, and is shown to be slower than n^−1/2 in case the estimator is tuned to perform consistent model selection. An impossibility result regarding estimation of the estimators’ distribution function is also provided.     

Difference based ridge and Liu type estimators in semiparametric regression models

We consider a difference based ridge regression estimator and a Liu type estimator of the regression parameters in the partial linear semiparametric regression model, y=Xβ+f+ε. Both estimators are analyzed and compared in the sense of mean-squared error. We consider the case of independent errors with equal variance and give conditions under which the proposed estimators are superior to the unbiased difference based estimation technique. We extend the results to account for heteroscedasticity and autocovariance in the error terms. Finally, we illustrate the performance of these estimators with an application to the determinants of electricity consumption in Germany.     

Confidence ellipsoids based on a general family of shrinkage estimators for a linear model with non-spherical disturbances

This paper considers a general family of Stein rule estimators for the coefficient vector of a linear regression model with nonspherical disturbances, and derives estimators for the Mean Squared Error (MSE) matrix, and risk under quadratic loss for this family of estimators. The confidence ellipsoids for the coefficient vector based on this family of estimators are proposed, and the performance of the confidence ellipsoids under the criterion of coverage probability and expected volumes is investigated. The results of a numerical simulation are presented to illustrate the theoretical findings, which could be applicable in the area of economic growth modeling.     

Hypothesis testing using affine linear estimators

The general linear hypothesis is usually tested by means of anF-statistic dependent on the least squares estimator. In this paper, a class of linear estimators is identified which can also serve as a basis for such anF-statistic. Conditions are derived under which thisF-statistic coincides with the usual one. This opens the possibility of constructing minimax-estimators which dominate LS with respect to risk, yielding the same test results.Support by Deutsche Forschungsgemeinschaft, Grant No. Tr 253/1-2 is gratefully acknowledged.     

Strong consistency of least-squares estimation in linear regression models with vague concepts

Linear regression models with vague concepts extend the classical single equation linear regression models by admitting observations in form of fuzzy subsets instead of real numbers. They have recently been introduced [cf. Krätschmer, Induktive statistik auf basis unscharfer meßkonzepte am beispiel linearer regressionsmodelle, Unpublished Habilitation Monograph, Faculty of Law and Economics of the University of Saarland, Saarbrücken, 2001] to improve the empirical meaningfulness of the relationship between the involved items by a more sensitive attention to the problems of data measurement, in particular the fundamental problem of adequacy. The parameters of such models are still real numbers, and a method of estimation can be applied which extends directly the ordinary least-squares method. This paper deals with some first asymptotic properties of estimators obtained by the method. Firstly, strong consistency will be shown, and secondly, the convergence rate will be investigated. The later result will be the starting point for a future study which will calculate the limit distributions of the estimators.     

On Bayes estimators with uniform priors on spheres and their comparative performance with maximum likelihood estimators for estimating bounded multivariate normal means

For independently distributed observables: X_i∼N(θ_i,σ²),i=1,…,p, we consider estimating the vector θ=(θ₁,…,θ_p)^′ with loss ‖d−θ‖² under the constraint , with known τ₁,…,τ_p,σ²,m. In comparing the risk performance of Bayesian estimators δ_α associated with uniform priors on spheres of radius α centered at (τ₁,…,τ_p) with that of the maximum likelihood estimator , we make use of Stein’s unbiased estimate of risk technique, Karlin’s sign change arguments, and a conditional risk analysis to obtain for a fixed (m,p) necessary and sufficient conditions on α for δ_α to dominate . Large sample determinations of these conditions are provided. Both cases where all such δ_α’s and cases where no such δ_α’s dominate are elicited. We establish, as a particular case, that the boundary uniform Bayes estimator δ_m dominates if and only if m≤k(p) with , improving on the previously known sufficient condition of Marchand and Perron (2001) [3] for which . Finally, we improve upon a universal dominance condition due to Marchand and Perron, by establishing that all Bayesian estimators δ_π with π spherically symmetric and supported on the parameter space dominate whenever m≤c₁(p) with .     

Existence conditions for the uniformly minimum risk unbiased estimators in a class of linear models

This paper studies the existence of the uniformly minimum risk unbiased (UMRU) estimators of parameters in a class of linear models with an error vector having multivariate normal distribution or t-distribution, which include the growth curve model, the extended growth curve model, the seemingly unrelated regression equations model, the variance components model, and so on. The necessary and sufficient existence conditions are established for UMRU estimators of the estimable linear functions of regression coefficients under convex losses and matrix losses, respectively. Under the (extended) growth curve model and the seemingly unrelated regression equations model with normality assumption, the conclusions given in the literature can be derived by applying the general results in this paper. For the variance components model, the necessary and sufficient existence conditions are reduced as terse forms.     

Asymptotic properties of computationally efficient alternative estimators for a class of multivariate normal models

Parameters of Gaussian multivariate models are often estimated using the maximum likelihood approach. In spite of its merits, this methodology is not practical when the sample size is very large, as, for example, in the case of massive georeferenced data sets. In this paper, we study the asymptotic properties of the estimators that minimize three alternatives to the likelihood function, designed to increase the computational efficiency. This is achieved by applying the information sandwich technique to expansions of the pseudo-likelihood functions as quadratic forms of independent normal random variables. Theoretical calculations are given for a first-order autoregressive time series and then extended to a two-dimensional autoregressive process on a lattice. We compare the efficiency of the three estimators to that of the maximum likelihood estimator as well as among themselves, using numerical calculations of the theoretical results and simulations.     

Thresholding projection estimators in functional linear models

We consider the problem of estimating the regression function in functional linear regression models by proposing a new type of projection estimators which combine dimension reduction and thresholding. The introduction of a threshold rule allows us to get consistency under broad assumptions as well as minimax rates of convergence under additional regularity hypotheses. We also consider the particular case of Sobolev spaces generated by the trigonometric basis which permits us to get easily mean squared error of prediction as well as estimators of the derivatives of the regression function. We prove that these estimators are minimax and rates of convergence are given for some particular cases.     

Moderate deviations of generalized method of moments and empirical likelihood estimators

This paper studies moderate deviation behaviors of the generalized method of moments and generalized empirical likelihood estimators for generalized estimating equations, where the number of equations can be larger than the number of unknown parameters. We consider two cases for the data generating probability measure: the model assumption and local contaminations or deviations from the model assumption. For both cases, we characterize the first-order terms of the moderate deviation error probabilities of these estimators. Our moderate deviation analysis complements the existing literature of the local asymptotic analysis and misspecification analysis for estimating equations, and is useful to evaluate power and robust properties of statistical tests for estimating equations which typically involve some estimators for nuisance parameters.