Bias-corrected AIC for selecting variables in multinomial logistic regression models

In this paper, we consider the bias correction of Akaike’s information criterion (AIC) for selecting variables in multinomial logistic regression models. For simplifying a formula of the bias-corrected AIC, we calculate the bias of the AIC to a risk function through the expectations of partial derivatives of the negative log-likelihood function. As a result, we can express the bias correction term of the bias-corrected AIC with only three matrices consisting of the second, third, and fourth derivatives of the negative log-likelihood function. By conducting numerical studies, we verify that the proposed bias-corrected AIC performs better than the crude AIC.     

Bayesian bandwidth estimation for a semi-functional partial linear regression model with unknown error density

In the context of semi-functional partial linear regression model, we study the problem of error density estimation. The unknown error density is approximated by a mixture of Gaussian densities with means being the individual residuals, and variance a constant parameter. This mixture error density has a form of a kernel density estimator of residuals, where the regression function, consisting of parametric and nonparametric components, is estimated by the ordinary least squares and functional Nadaraya–Watson estimators. The estimation accuracy of the ordinary least squares and functional Nadaraya–Watson estimators jointly depends on the same bandwidth parameter. A Bayesian approach is proposed to simultaneously estimate the bandwidths in the kernel-form error density and in the regression function. Under the kernel-form error density, we derive a kernel likelihood and posterior for the bandwidth parameters. For estimating the regression function and error density, a series of simulation studies show that the Bayesian approach yields better accuracy than the benchmark functional cross validation. Illustrated by a spectroscopy data set, we found that the Bayesian approach gives better point forecast accuracy of the regression function than the functional cross validation, and it is capable of producing prediction intervals nonparametrically.     

Conditional information criteria for selecting variables in linear mixed models

In this paper, we consider the problem of selecting the variables of the fixed effects in the linear mixed models where the random effects are present and the observation vectors have been obtained from many clusters. As the variable selection procedure, here we use the Akaike Information Criterion, AIC. In the context of the mixed linear models, two kinds of AIC have been proposed: marginal AIC and conditional AIC. In this paper, we derive three versions of conditional AIC depending upon different estimators of the regression coefficients and the random effects. Through the simulation studies, it is shown that the proposed conditional AIC’s are superior to the marginal and conditional AIC’s proposed in the literature in the sense of selecting the true model. Finally, the results are extended to the case when the random effects in all the clusters are of the same dimension but have a common unknown covariance matrix.     

Conditional and unconditional methods for selecting variables in linear mixed models

In the problem of selecting the explanatory variables in the linear mixed model, we address the derivation of the (unconditional or marginal) Akaike information criterion (AIC) and the conditional AIC (cAIC). The covariance matrices of the random effects and the error terms include unknown parameters like variance components, and the selection procedures proposed in the literature are limited to the cases where the parameters are known or partly unknown. In this paper, AIC and cAIC are extended to the situation where the parameters are completely unknown and they are estimated by the general consistent estimators including the maximum likelihood (ML), the restricted maximum likelihood (REML) and other unbiased estimators. We derive, related to AIC and cAIC, the marginal and the conditional prediction error criteria which select superior models in light of minimizing the prediction errors relative to quadratic loss functions. Finally, numerical performances of the proposed selection procedures are investigated through simulation studies.     

Estimating a reduced rank regression model for non-normal variables

A method using third order moments for estimating the regression coefficients as well as the latent state scores of the reduced-rank regression model when the latent variable(s) are non-normally distributed is presented in this paper. It is shown that the factor analysis type indeterminacy of the regression coefficient matrices is eliminated. A real life example of the proposed method is presented. Differences of this solution with the reduced-rank regression eigen solution are discussed.     

A new error in variables model for solving positive definite linear system using orthogonal matrix decompositions

The need to estimate a positive definite solution to an overdetermined linear system of equations with multiple right hand side vectors arises in several process control contexts. The coefficient and the right hand side matrices are respectively named data and target matrices. A number of optimization methods were proposed for solving such problems, in which the data matrix is unrealistically assumed to be error free. Here, considering error in measured data and target matrices, we present an approach to solve a positive definite constrained linear system of equations based on the use of a newly defined error function. To minimize the defined error function, we derive necessary and sufficient optimality conditions and outline a direct algorithm to compute the solution. We provide a comparison of our proposed approach and two existing methods, the interior point method and a method based on quadratic programming. Two important characteristics of our proposed method as compared to the existing methods are computing the solution directly and considering error both in data and target matrices. Moreover, numerical test results show that the new approach leads to smaller standard deviations of error entries and smaller effective rank as desired by control problems. Furthermore, in a comparative study, using the Dolan-Moré performance profiles, we show the approach to be more efficient.     

A fuzzy model for linear regression

Doklady Mathematics -     

Mean quadratic error of an estimate of splines of first order in a model of linear regression

We study the mean quadratic error of an estimate of splines of the first order, which is obtained by the method of least squares under the assumption that the data represents a superposition of proper values of a spline and a white noise. A quantitative formula for the quadratic mean error is found and its asymptotics is investigated.Translated from Ukrainskii Matematicheskii Zhurnal, Vol. 49, No. 3, pp. 429–432, March, 1991.     

Mean quadratic error of an estimate of splines of first order in a model of linear regression

We study the mean quadratic error of an estimate of splines of the first order, which is obtained by the method of least squares under the assumption that the data represents a superposition of proper values of a spline and a white noise. A quantitative formula for the quadratic mean error is found and its asymptotics is investigated.Translated from Ukrainskii Matematicheskii Zhurnal, Vol. 49, No. 3, pp. 429–432, March, 1991.     

A linear regression model for imprecise response

A linear regression model with imprecise response and p real explanatory variables is analyzed. The imprecision of the response variable is functionally described by means of certain kinds of fuzzy sets, the LR fuzzy sets. The LR fuzzy random variables are introduced to model usual random experiments when the characteristic observed on each result can be described with fuzzy numbers of a particular class, determined by 3 random values: the center, the left spread and the right spread. In fact, these constitute a natural generalization of the interval data. To deal with the estimation problem the space of the LR fuzzy numbers is proved to be isometric to a closed and convex cone of R³ with respect to a generalization of the most used metric for LR fuzzy numbers. The expression of the estimators in terms of moments is established, their limit distribution and asymptotic properties are analyzed and applied to the determination of confidence regions and hypothesis testing procedures. The results are illustrated by means of some case-studies.     

Reduced-rank regression for the multivariate linear model

The problem of estimating the regression coefficient matrix having known (reduced) rank for the multivariate linear model when both sets of variates are jointly stochastic is discussed. We show that this problem is related to the problem of deciding how many principal components or pairs of canonical variates to use in any practical situation. Under the assumption of joint normality of the two sets of variates, we give the asymptotic (large-sample) distributions of the various estimated reduced-rank regression coefficient matrices that are of interest. Approximate confidence bounds on the elements of these matrices are then suggested using either the appropriate asymptotic expressions or the jackknife technique.     

Empirical Bayes estimation in a multiple linear regression model

Summary Estimation of the vector β of the regression coefficients in a multiple linear regressionY=Xβ+ε is considered when β has a completely unknown and unspecified distribution and the error-vector ε has a multivariate standard normal distribution. The optimal estimator for β, which minimizes the overall mean squared error, cannot be constructed for use in practice. UsingX, Y and the information contained in the observation-vectors obtained fromn independent past experiences of the problem, (empirical Bayes) estimators for β are exhibited. These estimators are compared with the optimal estimator and are shown to be asymptotically optimal. Estimators asymptotically optimal with rates nearO(n ⁻¹) are constructed. Supported in part by a Natural Sciences and Engineering Research Council of Canada grant.     

Functional-coefficient partially linear regression model

In this paper, the functional-coefficient partially linear regression (FCPLR) model is proposed by combining nonparametric and functional-coefficient regression (FCR) model. It includes the FCR model and the nonparametric regression (NPR) model as its special cases. It is also a generalization of the partially linear regression (PLR) model obtained by replacing the parameters in the PLR model with some functions of the covariates. The local linear technique and the integrated method are employed to give initial estimators of all functions in the FCPLR model. These initial estimators are asymptotically normal. The initial estimator of the constant part function shares the same bias as the local linear estimator of this function in the univariate nonparametric model, but the variance of the former is bigger than that of the latter. Similarly, initial estimators of every coefficient function share the same bias as the local linear estimates in the univariate FCR model, but the variance of the former is bigger than that of the latter. To decrease the variance of the initial estimates, a one-step back-fitting technique is used to obtain the improved estimators of all functions. The improved estimator of the constant part function has the same asymptotic normality property as the local linear nonparametric regression for univariate data. The improved estimators of the coefficient functions have the same asymptotic normality properties as the local linear estimates in FCR model. The bandwidths and the smoothing variables are selected by a data-driven method. Both simulated and real data examples related to nonlinear time series modeling are used to illustrate the applications of the FCPLR model.     

Minimum distance estimation in linear regression with unknown error distributions

This paper discusses minimum distance (m.d.) estimators of the paramter vector in the multiple linear regression model when the distributions of errors are unknown. These estimators are defined in terms of L₂-distances involving certain weighted empirical processes. Their finite sample properties and asymptotic behavior under heteroscedastic, symmetric and asymmetric errors are discussed. Some robustness properties of these estimators are also studied.     

Corrected version of AIC for selecting multivariate normal linear regression models in a general nonnormal case

This paper deals with the bias reduction of Akaike information criterion (AIC) for selecting variables in multivariate normal linear regression models when the true distribution of observation is an unknown nonnormal distribution. We propose a corrected version of AIC which is partially constructed by the jackknife method and is adjusted to the exact unbiased estimator of the risk when the candidate model includes the true model. It is pointed out that the influence of nonnormality in the bias of our criterion is smaller than the ones in AIC and TIC. We verify that our criterion is better than the AIC, TIC and EIC by conducting numerical experiments.     

Reconstruction of the error distribution in the linear regression scheme

Under a certain condition, the characterization of the additive type of an error distribution by the distribution of a maximal invariant is proved in the linear regression scheme. With an added condition on the design matrix, the possibility of reconstructing the error distribution itself is proved.Translated from Zapiski Nauchnykh Seminarov Leningradskogo Otdeleniya Matematicheskogo Instituta im. V. A. Steklova AN SSSR, Vol. 87, pp. 36–40, 1979.The author is grateful to L. B. Klebanov for his interest in this article.     

Nonlinear wavelet smoothing of error density in a semiparametric regression model

In this paper,a semlparametrie resresaion model in which errors are i. i. d random variables from an unknown density f( ) is considered. Based on Hall et al. (1995),a nonlinear wavelet estimation of f( ) without restrictions of continuity everywhere on f( ) is given,and the convergence rate of the estimators in L2 is obtained.