Limit distributions of least squares estimators in linear regression models with vague concepts

Linear regression models with vague concepts extend the classical single equation linear regression models by admitting observations in form of fuzzy subsets instead of real numbers. They have lately been introduced (cf. [V. Krätschmer, Induktive Statistik auf Basis unscharfer Meßkonzepte am Beispiel linearer Regressionsmodelle, unpublished postdoctoral thesis, Faculty of Law and Economics of the University of Saarland, Saarbrücken, 2001; V. Krätschmer, Least squares estimation in linear regression models with vague concepts, Fuzzy Sets and Systems, accepted for publication]) to improve the empirical meaningfulness of the relationships between the involved items by a more sensitive attention to the problems of data measurement, in particular, the fundamental problem of adequacy. The parameters of such models are still real numbers, and a method of estimation can be applied which extends directly the ordinary least squares method. In another recent contribution (cf. [V. Krätschmer, Strong consistency of least squares estimation in linear regression models with vague concepts, J. Multivar. Anal., accepted for publication]) strong consistency and -consistency of this generalized least squares estimation have been shown. The aim of the paper is to complete these results by an investigation of the limit distributions of the estimators. It turns out that the classical results can be transferred, in some cases even asymptotic normality holds.     

On linear models with long memory and heavy-tailed errors

We consider the robust estimation of regression parameters in linear models with long memory and heavy-tailed errors. Asymptotic Bahadur-type representations of robust estimates are developed and their limiting distributions are obtained. It is shown that the limiting distributions are very different from those obtained under short memory. A simulation study is carried out to compare the performance of various asymptotic representations.     

Strong consistency of least-squares estimation in linear regression models with vague concepts

Linear regression models with vague concepts extend the classical single equation linear regression models by admitting observations in form of fuzzy subsets instead of real numbers. They have recently been introduced [cf. Krätschmer, Induktive statistik auf basis unscharfer meßkonzepte am beispiel linearer regressionsmodelle, Unpublished Habilitation Monograph, Faculty of Law and Economics of the University of Saarland, Saarbrücken, 2001] to improve the empirical meaningfulness of the relationship between the involved items by a more sensitive attention to the problems of data measurement, in particular the fundamental problem of adequacy. The parameters of such models are still real numbers, and a method of estimation can be applied which extends directly the ordinary least-squares method. This paper deals with some first asymptotic properties of estimators obtained by the method. Firstly, strong consistency will be shown, and secondly, the convergence rate will be investigated. The later result will be the starting point for a future study which will calculate the limit distributions of the estimators.     

On Monte Carlo methods for Bayesian multivariate regression models with heavy-tailed errors

We consider Bayesian analysis of data from multivariate linear regression models whose errors have a distribution that is a scale mixture of normals. Such models are used to analyze data on financial returns, which are notoriously heavy-tailed. Let π denote the intractable posterior density that results when this regression model is combined with the standard non-informative prior on the unknown regression coefficients and scale matrix of the errors. Roughly speaking, the posterior is proper if and only if n≥d+k, where n is the sample size, d is the dimension of the response, and k is number of covariates. We provide a method of making exact draws from π in the special case where n=d+k, and we study Markov chain Monte Carlo (MCMC) algorithms that can be used to explore π when n>d+k. In particular, we show how the Haar PX-DA technology studied in Hobert and Marchev (2008) [11] can be used to improve upon Liu’s (1996) [7] data augmentation (DA) algorithm. Indeed, the new algorithm that we introduce is theoretically superior to the DA algorithm, yet equivalent to DA in terms of computational complexity. Moreover, we analyze the convergence rates of these MCMC algorithms in the important special case where the regression errors have a Student’s t distribution. We prove that, under conditions on n, d, k, and the degrees of freedom of the t distribution, both algorithms converge at a geometric rate. These convergence rate results are important from a practical standpoint because geometric ergodicity guarantees the existence of central limit theorems which are essential for the calculation of valid asymptotic standard errors for MCMC based estimates.     

Biased estimation and hypothesis testing in linear regression

We consider a test of the simple hypothesis =₀ based on some biased estimator. Under a certain condition the corresponding test statistic coincides with the usualF-statistic based on the least squares estimator. Surprisingly, this condition is met by several well-known biased estimators.     

Estimators of scale parameters in linear regression

This note discusses the asymptotic distribution of two scale and location invariant estimators of two scale parameters in the multiple linear regression model. Both of these estimators need an initial estimator of the regression parameter vector. The asymptotic distribution of one of these estimators does not depend on this initial estimator. Both of these estimators are useful in the computation of scale and translation invariant adaptive estimators and M-estimators of the regression parameter vector.     

Bayesian modeling of several covariance matrices and some results on propriety of the posterior for linear regression with correlated and/or heterogeneous errors

We explore simultaneous modeling of several covariance matrices across groups using the spectral (eigenvalue) decomposition and modified Cholesky decomposition. We introduce several models for covariance matrices under different assumptions about the mean structure. We consider ‘dependence’ matrices, which tend to have many parameters, as constant across groups and/or parsimoniously modeled via a regression formulation. For ‘variances’, we consider both unrestricted across groups and more parsimoniously modeled via log-linear models. In all these models, we explore the propriety of the posterior when improper priors are used on the mean and ‘variance’ parameters (and in some cases, on components of the ‘dependence’ matrices). The models examined include several common Bayesian regression models, whose propriety has not been previously explored, as special cases. We propose a simple approach to weaken the assumption of constant dependence matrices in an automated fashion and describe how to compute Bayes factors to test the hypothesis of constant ‘dependence’ across groups. The models are applied to data from two longitudinal clinical studies.     

Partial sum process to check regression models with multiple correlated response: With an application for testing a change-point in profile data

We consider regression models with multiple correlated responses for each design point. Under the null hypothesis, a linear regression is assumed. For the least-squares residuals of this linear regression, we establish the limit of the partial sums. This limit is a projection on a certain subspace of the reproducing Kernel Hilbert space of a multivariate Brownian motion. Based on this limit, we propose a significance test of Kolmogorov-Smirnov type to test the null hypothesis and show that this result can be used to study a change-point problem in the case of linear profile data (panel data). We compare our proposed method, which does not rely on any distributional assumptions, with the likelihood ratio test in a simulation study.     

Best linear unbiased prediction for linear combinations in general mixed linear models

The general mixed linear model can be written as . In this paper, we mainly deal with two problems. Firstly, the problem of predicting a general linear combination of fixed effects and realized values of random effects in a general mixed linear model is considered and an explicit representation of the best linear unbiased predictor (BLUP) is derived. In addition, we apply the resulting conclusion to several special models and offer an alternative to characterization of BLUP. Secondly, we recall the notion of linear sufficiency and consider it as regards the BLUP problem and characterize it in several different ways. Further, we study the concepts of linear sufficiency, linear minimal sufficiency and linear completeness, and give relations among them. Finally, four concluding remarks are given.     

Theory and inference for regression models with missing responses and covariates

In this paper, we carry out an in-depth theoretical investigation for inference with missing response and covariate data for general regression models. We assume that the missing data are missing at random (MAR) or missing completely at random (MCAR) throughout. Previous theoretical investigations in the literature have focused only on missing covariates or missing responses, but not both. Here, we consider theoretical properties of the estimates under three different estimation settings: complete case (CC) analysis, a complete response (CR) analysis that involves an analysis of those subjects with only completely observed responses, and the all case (AC) analysis, which is an analysis based on all of the cases. Under each scenario, we derive general expressions for the likelihood and devise estimation schemes based on the EM algorithm. We carry out a theoretical investigation of the three estimation methods in the normal linear model and analytically characterize the loss of information for each method, as well as derive and compare the asymptotic variances for each method assuming the missing data are MAR or MCAR. In addition, a theoretical investigation of bias for the CC method is also carried out. A simulation study and real dataset are given to illustrate the methodology.     

Nonnegative quadratic estimation of the mean squared errors of minimax estimators in the linear regression model

In this paper, the problem of nonnegative quadratic estimation of the mean squared errors of minimax estimators of in the linear regression modelE(y)=X, VAR(y) = ² is discussed. An explicit formula for the admissible nonnegative minimum biased estimator is given. Some applications to one-way classification model are also considered.     

Composing the cumulative quantile regression function and the Goldie concentration curve

The model we discuss in this paper deals with inequality in distribution in the presence of a covariate. To elucidate that dependence, we propose to consider the composition of the cumulative quantile regression (CQR) function and the Goldie concentration curve, the standardized counterpart of which gives a fraction to fraction plot of the response and the covariate. It has the merit of enhancing the visibility of inequality in distribution when the latter is present. We shall examine the asymptotic properties of the corresponding empirical estimator. The associated empirical process involves a randomly stopped partial sum process of induced order statistics. Strong Gaussian approximations of the processes are constructed. The result forms the basis for the asymptotic theory of functional statistics based on these processes.     

Estimation using the linear regression model with incomplete ellipsoidal restrictions

Using the linear regression model with incomplete ellipsoidal restrictions, it is shown that the known Kuks-Olman estimator is still an appropriate choice.     

Risk and Pitman closeness properties of feasible generalized double k-class estimators in linear regression models with non-spherical disturbances under balanced loss function

In this article, a family of feasible generalized double k-class estimator in a linear regression model with non-spherical disturbances is considered. The performance of this estimator is judged with feasible generalized least-squares and feasible generalized Stein-rule estimators under balanced loss function using the criteria of quadratic risk and general Pitman closeness. A Monte-Carlo study investigates the finite sample properties of several estimators arising from the family of feasible double k-class estimators.     

Simultaneous change point analysis and variable selection in a regression problem

In this paper, an information-based criterion is proposed for carrying out change point analysis and variable selection simultaneously in linear models with a possible change point. Under some weak conditions, this criterion is shown to be strongly consistent in the sense that with probability one, it chooses the smallest true model for large n. Its byproducts include strongly consistent estimates of the regression coefficients regardless if there is a change point. In case that there is a change point, its byproducts also include a strongly consistent estimate of the change point parameter. In addition, an algorithm is given which has significantly reduced the computation time needed by the proposed criterion for the same precision. Results from a simulation study are also presented.     

Statistical inference for panel data semiparametric partially linear regression models with heteroscedastic errors

We consider a panel data semiparametric partially linear regression model with an unknown parameter vector for the linear parametric component, an unknown nonparametric function for the nonlinear component, and a one-way error component structure which allows unequal error variances (referred to as heteroscedasticity). We develop procedures to detect heteroscedasticity and one-way error component structure, and propose a weighted semiparametric least squares estimator (WSLSE) of the parametric component in the presence of heteroscedasticity and/or one-way error component structure. This WSLSE is asymptotically more efficient than the usual semiparametric least squares estimator considered in the literature. The asymptotic properties of the WSLSE are derived. The nonparametric component of the model is estimated by the local polynomial method. Some simulations are conducted to demonstrate the finite sample performances of the proposed testing and estimation procedures. An example of application on a set of panel data of medical expenditures in Australia is also illustrated.     

Efficiency of a Liu-type estimator in semiparametric regression models

In this paper we consider the semiparametric regression model, y=Xβ+f+ε. Recently, Hu [11] proposed ridge regression estimator in a semiparametric regression model. We introduce a Liu-type (combined ridge-Stein) estimator (LTE) in a semiparametric regression model. Firstly, Liu-type estimators of both β and f are attained without a restrained design matrix. Secondly, the LTE estimator of β is compared with the two-step estimator in terms of the mean square error. We describe the almost unbiased Liu-type estimator in semiparametric regression models. The almost unbiased Liu-type estimator is compared with the Liu-type estimator in terms of the mean squared error matrix. A numerical example is provided to show the performance of the estimators.     

Statistical inference of minimum BD estimators and classifiers for varying-dimensional models

Stochastic modeling for large-scale datasets usually involves a varying-dimensional model space. This paper investigates the asymptotic properties, when the number of parameters grows with the available sample size, of the minimum- estimators and classifiers under a broad and important class of Bregman divergence (), which encompasses nearly all of the commonly used loss functions in the regression analysis, classification procedures and machine learning literature. Unlike the maximum likelihood estimators which require the joint likelihood of observations, the minimum-BD estimators are useful for a range of models where the joint likelihood is unavailable or incomplete. Statistical inference tools developed for the class of large dimensional minimum- estimators and related classifiers are evaluated via simulation studies, and are illustrated by analysis of a real dataset.