Selection of smoothing parameters in<Emphasis Type="Italic">B</Emphasis>-spline nonparametric regression models using information criteria

We consider the use ofB-spline nonparametric regression models estimated by the maximum penalized likelihood method for extracting information from data with complex nonlinear structure. Crucial points inB-spline smoothing are the choices of a smoothing parameter and the number of basis functions, for which several selectors have been proposed based on cross-validation and Akaike information criterion known as AIC. It might be however noticed that AIC is a criterion for evaluating models estimated by the maximum likelihood method, and it was derived under the assumption that the ture distribution belongs to the specified parametric model. In this paper we derive information criteria for evaluatingB-spline nonparametric regression models estimated by the maximum penalized likelihood method in the context of generalized linear models under model misspecification. We use Monte Carlo experiments and real data examples to examine the properties of our criteria including various selectors proposed previously.     

Conditional and unconditional methods for selecting variables in linear mixed models

In the problem of selecting the explanatory variables in the linear mixed model, we address the derivation of the (unconditional or marginal) Akaike information criterion (AIC) and the conditional AIC (cAIC). The covariance matrices of the random effects and the error terms include unknown parameters like variance components, and the selection procedures proposed in the literature are limited to the cases where the parameters are known or partly unknown. In this paper, AIC and cAIC are extended to the situation where the parameters are completely unknown and they are estimated by the general consistent estimators including the maximum likelihood (ML), the restricted maximum likelihood (REML) and other unbiased estimators. We derive, related to AIC and cAIC, the marginal and the conditional prediction error criteria which select superior models in light of minimizing the prediction errors relative to quadratic loss functions. Finally, numerical performances of the proposed selection procedures are investigated through simulation studies.     

A Comparative Investigation on Model Selection in Independent Factor Analysis

With uncorrelated Gaussian factors extended to mutually independent factors beyond Gaussian, the conventional factor analysis is extended to what is recently called independent factor analysis. Typically, it is called binary factor analysis (BFA) when the factors are binary and called non-Gaussian factor analysis (NFA) when the factors are from real non-Gaussian distributions. A crucial issue in both BFA and NFA is the determination of the number of factors. In the literature of statistics, there are a number of model selection criteria that can be used for this purpose. Also, the Bayesian Ying-Yang (BYY) harmony learning provides a new principle for this purpose. This paper further investigates BYY harmony learning in comparison with existing typical criteria, including Akaik’s information criterion (AIC), the consistent Akaike’s information criterion (CAIC), the Bayesian inference criterion (BIC), and the cross-validation (CV) criterion on selection of the number of factors. This comparative study is made via experiments on the data sets with different sample sizes, data space dimensions, noise variances, and hidden factors numbers. Experiments have shown that for both BFA and NFA, in most cases BIC outperforms AIC, CAIC, and CV while the BYY criterion is either comparable with or better than BIC. In consideration of the fact that the selection by these criteria has to be implemented at the second stage based on a set of candidate models which have to be obtained at the first stage of parameter learning, while BYY harmony learning can provide not only a new class of criteria implemented in a similar way but also a new family of algorithms that perform parameter learning at the first stage with automated model selection, BYY harmony learning is more preferred since computing costs can be saved significantly.     

An information criterion for model selection with missing data via complete-data divergence

We derive an information criterion to select a parametric model of complete-data distribution when only incomplete or partially observed data are available. Compared with AIC, our new criterion has an additional penalty term for missing data, which is expressed by the Fisher information matrices of complete data and incomplete data. We prove that our criterion is an asymptotically unbiased estimator of complete-data divergence, namely the expected Kullback–Leibler divergence between the true distribution and the estimated distribution for complete data, whereas AIC is that for the incomplete data. The additional penalty term of our criterion for missing data turns out to be only half the value of that in previously proposed information criteria PDIO and AICcd. The difference in the penalty term is attributed to the fact that our criterion is derived under a weaker assumption. A simulation study with the weaker assumption shows that our criterion is unbiased while the other two criteria are biased. In addition, we review the geometrical view of alternating minimizations of the EM algorithm. This geometrical view plays an important role in deriving our new criterion.     

Bias-corrected AIC for selecting variables in multinomial logistic regression models

In this paper, we consider the bias correction of Akaike’s information criterion (AIC) for selecting variables in multinomial logistic regression models. For simplifying a formula of the bias-corrected AIC, we calculate the bias of the AIC to a risk function through the expectations of partial derivatives of the negative log-likelihood function. As a result, we can express the bias correction term of the bias-corrected AIC with only three matrices consisting of the second, third, and fourth derivatives of the negative log-likelihood function. By conducting numerical studies, we verify that the proposed bias-corrected AIC performs better than the crude AIC.     

Bootstrapping Log Likelihood and EIC, an Extension of AIC

Akaike (1973, 2nd International Symposium on Information Theory, 267-281,Akademiai Kiado, Budapest) proposed AIC as an estimate of the expected loglikelihood to evaluate the goodness of models fitted to a given set of data.The introduction of AIC has greatly widened the range of application ofstatistical methods. However, its limit lies in the point that it can beapplied only to the cases where the parameter estimation are performed bythe maximum likelihood method. The derivation of AIC is based on theassessment of the effect of data fluctuation through the asymptoticnormality of MLE. In this paper we propose a new information criterion EICwhich is constructed by employing the bootstrap method to simulate the datafluctuation. The new information criterion, EIC, is regarded as an extensionof AIC. The performance of EIC is demonstrated by some numerical examples.     

Minimizing variable selection criteria by Markov chain Monte Carlo

Regression models with a large number of predictors arise in diverse fields of social sciences and natural sciences. For proper interpretation, we often would like to identify a smaller subset of the variables that shows the strongest information. In such a large size of candidate predictors setting, one would encounter a computationally cumbersome search in practice by optimizing some criteria for selecting variables, such as AIC, \(C_{P}\) and BIC, through all possible subsets. In this paper, we present two efficient optimization algorithms vis Markov chain Monte Carlo (MCMC) approach for searching the global optimal subset. Simulated examples as well as one real data set exhibit that our proposed MCMC algorithms did find better solutions than other popular search methods in terms of minimizing a given criterion.     

AIC type statistics for discretely observed ergodic diffusion processes

We consider the model selection problem for ergodic diffusion processes based on sampled data. The adaptive estimators for parameters of drift and diffusion coefficients are used in order to construct Akaike’s information criterion (AIC) type model selection statistics. Asymptotic properties of our proposed criteria are given for three kinds of the adaptive estimators.     

A Bayesian Estimate of the Risk of Tick-Borne Diseases

The paper considers the problem of estimating the risk of a tick-borne disease in a given region. A large set of epidemiological data is evaluated, including the point pattern of collected cases, the population map and covariates, i.e. explanatory variables of geographical nature, obtained from GIS.The methodology covers the choice of those covariates which influence the risk of infection most. Generalized linear models are used and AIC criterion yields the decision. Further, an empirical Bayesian approach is used to estimate the parameters of the risk model. Statistical properties of the estimators are investigated. Finally, a comparison with earlier results is discussed from the point of view of statistical disease mapping.     

A multistage algorithm for best-subset model selection based on the Kullback–Leibler discrepancy

The selection of a best-subset regression model from a candidate family is a common problem that arises in many analyses. The Akaike information criterion (AIC) and the corrected AIC (\(\text {AIC}_c\)) are frequently used for this purpose. AIC and \(\text {AIC}_c\) are designed to estimate the expected Kullback–Leibler discrepancy. For best-subset selection, both AIC and \(\text {AIC}_c\) are negatively biased, and the use of either criterion will lead to the selection of overfitted models. To correct for this bias, we introduce an “improved” AIC variant, \(\text {AIC}_i\), which has a penalty term evaluated using Monte Carlo simulation. A multistage model selection procedure \(\text {AIC}_{\text {aps}}\), which utilizes \(\text {AIC}_i\), is proposed for best-subset selection. Simulation studies are compiled to compare the performances of the different model selection methods.     

Estimation des ordres de modèles ARMA faibles multivariés

In this Note, we consider the problem of order selection of vector autoregressive moving-average (VARMA) models under the assumption that the errors are uncorrelated, but not necessarily independent. These models are called weak VARMA by opposition to the standard VARMA models, also called strong VARMA models, in which the error terms are supposed to be iid. This selection is based on minimizing an information criterion, especially that introduced by Akaike. The theoretical foundations of the Akaike information criterion (AIC) are not more established when the iid assumption on the noise is relaxed. We propose a modified AIC criterion, and which may be very different from the standard AIC criterion.     

An efficient method for feature selection in linear regression based on an extended Akaike’s information criterion

A method for feature selection in linear regression based on an extension of Akaike’s information criterion is proposed. The use of classical Akaike’s information criterion (AIC) for feature selection assumes the exhaustive search through all the subsets of features, which has unreasonably high computational and time cost. A new information criterion is proposed that is a continuous extension of AIC. As a result, the feature selection problem is reduced to a smooth optimization problem. An efficient procedure for solving this problem is derived. Experiments show that the proposed method enables one to efficiently select features in linear regression. In the experiments, the proposed procedure is compared with the relevance vector machine, which is a feature selection method based on Bayesian approach. It is shown that both procedures yield similar results. The main distinction of the proposed method is that certain regularization coefficients are identical zeros. This makes it possible to avoid the underfitting effect, which is a characteristic feature of the relevance vector machine. A special case (the so-called nondiagonal regularization) is considered in which both methods are identical.     

Simultaneous variable selection for heteroscedastic regression models

In this paper, we propose a new criterion, named PICa, to simultaneously select explanatory variables in the mean model and variance model in heteroscedastic linear models based on the model structure. We show that the new criterion can select the true mean model and a correct variance model with probability tending to 1 under mild conditions. Simulation studies and a real example are presented to evaluate the new criterion, and it turns out that the proposed approach performs well.     

Hierarchical nonparametric survival modeling for demand forecasting with fragmented categorical covariates

This paper addresses the problem of data fragmentation when incorporating imbalanced categorical covariates in nonparametric survival models. The problem arises in an application of demand forecasting where certain categorical covariates are important explanatory factors for the diversity of survival patterns but are severely imbalanced in the sense that a large percentage of data segments defined by these covariates have very small sample sizes. Two general approaches, called the class‐based approach and the fusion‐based approach, are proposed to handle the problem. Both reply on judicious utilization of a data segment hierarchy defined by the covariates. The class‐based approach allows certain segments in the hierarchy to have their private survival functions and aggregates the others to share a common survival function. The fusion‐based approach allows all survival functions to borrow and share information from all segments based on their positions in the hierarchy. A nonparametric Bayesian estimator with Dirichlet process priors provides the data‐sharing mechanism in the fusion‐based approach. The hyperparameters in the priors are treated as fixed quantities and learned from data by taking advantage of the data segment hierarchy. The proposed methods are motivated and validated by a case study with real‐world data from an operation of software development service.     

Prognostic of glial tumours: Presentation of the problem with solution

A study of glial tumours involving 192 cases is presented. Different issues are addressed: (i) the interrelationships between the histological variables, (ii) the problem of the prediction of the survival time, (iii) the causal role of the variables in the progress of the disease. We propose a three-level grade which can be defined alternatively with perivascular lymphocites or with the signs necrosis and neovascularization. We constructed a predictive model based on the Cox model in which the variables were chosen according to Akaike's criterion. In the explanatory analysis we dropped the variables which could be considered as consequences rather than causes of the disease and we first tested groups of variables (factors): we found that age, the topology and the histology of the tumour were explanatory.     

Clusterwise analysis for multiblock component methods

Multiblock component methods are applied to data sets for which several blocks of variables are measured on a same set of observations with the goal to analyze the relationships between these blocks of variables. In this article, we focus on multiblock component methods that integrate the information found in several blocks of explanatory variables in order to describe and explain one set of dependent variables. In the following, multiblock PLS and multiblock redundancy analysis are chosen, as particular cases of multiblock component methods when one set of variables is explained by a set of predictor variables that is organized into blocks. Because these multiblock techniques assume that the observations come from a homogeneous population they will provide suboptimal results when the observations actually come from different populations. A strategy to palliate this problem—presented in this article—is to use a technique such as clusterwise regression in order to identify homogeneous clusters of observations. This approach creates two new methods that provide clusters that have their own sets of regression coefficients. This combination of clustering and regression improves the overall quality of the prediction and facilitates the interpretation. In addition, the minimization of a well-defined criterion—by means of a sequential algorithm—ensures that the algorithm converges monotonously. Finally, the proposed method is distribution-free and can be used when the explanatory variables outnumber the observations within clusters. The proposed clusterwise multiblock methods are illustrated with of a simulation study and a (simulated) example from marketing.     

Selecting mixed-effects models based on a generalized information criterion

The generalized information criterion (GIC) proposed by Rao and Wu [A strongly consistent procedure for model selection in a regression problem, Biometrika 76 (1989) 369-374] is a generalization of Akaike's information criterion (AIC) and the Bayesian information criterion (BIC). In this paper, we extend the GIC to select linear mixed-effects models that are widely applied in analyzing longitudinal data. The procedure for selecting fixed effects and random effects based on the extended GIC is provided. The asymptotic behavior of the extended GIC method for selecting fixed effects is studied. We prove that, under mild conditions, the selection procedure is asymptotically loss efficient regardless of the existence of a true model and consistent if a true model exists. A simulation study is carried out to empirically evaluate the performance of the extended GIC procedure. The results from the simulation show that if the signal-to-noise ratio is moderate or high, the percentages of choosing the correct fixed effects by the GIC procedure are close to one for finite samples, while the procedure performs relatively poorly when it is used to select random effects.     

Statistical inferences based on INID progressively type II censored order statistics

Suppose that the failure times of the units placed on a life-testing experiment are independent but nonidentically distributed random variables. Under progressively type II censoring scheme, distributional properties of the proposed random variables are presented and some inferences are made. Assuming that the random variables come from a proportional hazard rate model, the formulas are simplified and also the amount of Fisher information about the common parameters of this family is calculated. The results are also extended to a fixed covariates model. The performance of the proposed procedure is investigated via a real data set. Some numerical computations are also presented to study the effect of the proportionality rates in view of the Fisher information criterion. Finally, some concluding remarks are stated.     

Conditional information criteria for selecting variables in linear mixed models

In this paper, we consider the problem of selecting the variables of the fixed effects in the linear mixed models where the random effects are present and the observation vectors have been obtained from many clusters. As the variable selection procedure, here we use the Akaike Information Criterion, AIC. In the context of the mixed linear models, two kinds of AIC have been proposed: marginal AIC and conditional AIC. In this paper, we derive three versions of conditional AIC depending upon different estimators of the regression coefficients and the random effects. Through the simulation studies, it is shown that the proposed conditional AIC’s are superior to the marginal and conditional AIC’s proposed in the literature in the sense of selecting the true model. Finally, the results are extended to the case when the random effects in all the clusters are of the same dimension but have a common unknown covariance matrix.     

基于Pairwise惩罚方法的面板数据模型结构变点估计

面板数据模型在经济、生物、统计等领域有着广泛的应用。经典的面板数据模型假设解释变量系数不随时间变化。然而在现实中,解释变量系数可能会因多种因素的影响而存在多重未知的结构变点。本文假设交互固定效应面板数据模型中含有多重未知的结构变点。研究发现通过Pairwise惩罚的参数估计方法在目标函数中增加对相邻时间解释变量系数的惩罚项,能够同时进行参数估计和结构变点诊断。蒙特卡洛模拟结果显示,不管是否存在同方差假设,该方法估计的解释变量系数均偏差较小且结构变点诊断错误率低。