Prediction error criterion for selecting variables in a linear regression model

Several criteria, such as CV, C _p , AIC, CAIC, and MAIC, are used for selecting variables in linear regression models. It might be noted that C _p has been proposed as an estimator of the expected standardized prediction error, although the target risk function of CV might be regarded as the expected prediction error R _PE. On the other hand, the target risk function of AIC, CAIC, and MAIC is the expected log-predictive likelihood. In this paper, we propose a prediction error criterion, PE, which is an estimator of the expected prediction error R _PE. Consequently, it is also a competitor of CV. Results of this study show that PE is an unbiased estimator when the true model is contained in the full model. The property is shown without the assumption of normality. In fact, PE is demonstrated as more faithful for its risk function than CV. The prediction error criterion PE is extended to the multivariate case. Furthermore, using simulations, we examine some peculiarities of all these criteria.     

An information criterion for model selection with missing data via complete-data divergence

We derive an information criterion to select a parametric model of complete-data distribution when only incomplete or partially observed data are available. Compared with AIC, our new criterion has an additional penalty term for missing data, which is expressed by the Fisher information matrices of complete data and incomplete data. We prove that our criterion is an asymptotically unbiased estimator of complete-data divergence, namely the expected Kullback–Leibler divergence between the true distribution and the estimated distribution for complete data, whereas AIC is that for the incomplete data. The additional penalty term of our criterion for missing data turns out to be only half the value of that in previously proposed information criteria PDIO and AICcd. The difference in the penalty term is attributed to the fact that our criterion is derived under a weaker assumption. A simulation study with the weaker assumption shows that our criterion is unbiased while the other two criteria are biased. In addition, we review the geometrical view of alternating minimizations of the EM algorithm. This geometrical view plays an important role in deriving our new criterion.     

Continuous spin models on annealed generalized random graphs

We study Gibbs distributions of spins taking values in a general compact Polish space, interacting via a pair potential along the edges of a generalized random graph with a given asymptotic weight distribution $P$, obtained by annealing over the random graph distribution.First we prove a variational formula for the corresponding annealed pressure and provide criteria for absence of phase transitions in the general case.We furthermore study classes of models with second order phase transitions which include rotation-invariant models on spheres and models on intervals, and classify their critical exponents. We find critical exponents which are modified relative to the corresponding mean-field values when $P$ becomes too heavy-tailed, in which case they move continuously with the tail-exponent of $P$. For large classes of models they are the same as for the Ising model treated in Dommers et al. (2016). On the other hand, we provide conditions under which the model is in a different universality class, and construct an explicit example of such a model on the interval.     

A general framework for frequentist model averaging In Celebration of Professor Lincheng Zhao's 75th Birthday

Model selection strategies have been routinely employed to determine a model for data analysis in statistics, and further study and inference then often proceed as though the selected model were the true model that were known a priori. Model averaging approaches, on the other hand, try to combine estimators for a set of candidate models. Specifically, instead of deciding which model is the 'right' one, a model averaging approach suggests to fit a set of candidate models and average over the estimators using data adaptive weights.In this paper we establish a general frequentist model averaging framework that does not set any restrictions on the set of candidate models. It broaden, the scope of the existing methodologies under the frequentist model averaging development. Assuming the data is from an unknown model, we derive the model averaging estimator and study its limiting distributions and related predictions while taking possible modeling biases into account.We propose a set of optimal weights to combine the individual estimators so that the expected mean squared error of the average estimator is minimized. Simulation studies are conducted to compare the performance of the estimator with that of the existing methods. The results show the benefits of the proposed approach over traditional model selection approaches as well as existing model averaging methods.     

Corrected version of AIC for selecting multivariate normal linear regression models in a general nonnormal case

This paper deals with the bias reduction of Akaike information criterion (AIC) for selecting variables in multivariate normal linear regression models when the true distribution of observation is an unknown nonnormal distribution. We propose a corrected version of AIC which is partially constructed by the jackknife method and is adjusted to the exact unbiased estimator of the risk when the candidate model includes the true model. It is pointed out that the influence of nonnormality in the bias of our criterion is smaller than the ones in AIC and TIC. We verify that our criterion is better than the AIC, TIC and EIC by conducting numerical experiments.     

一类线性模型下最优线性无偏估计的存在性

考虑包含如方差分量模型、似乎不相关回归方程模型、增长曲线模型和扩充的增长曲线模型等众多常见模型的一类较广泛的线性模型。对模型中误差向量的分布不作假定时,给出了在二次损失或矩阵损失下存在回归系数的线性可估函数的一致最小风险线性无偏估计的充分必要条件。     

Twenty years of linear programming based portfolio optimization

Markowitz formulated the portfolio optimization problem through two criteria: the expected return and the risk, as a measure of the variability of the return. The classical Markowitz model uses the variance as the risk measure and is a quadratic programming problem. Many attempts have been made to linearize the portfolio optimization problem. Several different risk measures have been proposed which are computationally attractive as (for discrete random variables) they give rise to linear programming (LP) problems. About twenty years ago, the mean absolute deviation (MAD) model drew a lot of attention resulting in much research and speeding up development of other LP models. Further, the LP models based on the conditional value at risk (CVaR) have a great impact on new developments in portfolio optimization during the first decade of the 21st century. The LP solvability may become relevant for real-life decisions when portfolios have to meet side constraints and take into account transaction costs or when large size instances have to be solved. In this paper we review the variety of LP solvable portfolio optimization models presented in the literature, the real features that have been modeled and the solution approaches to the resulting models, in most of the cases mixed integer linear programming (MILP) models. We also discuss the impact of the inclusion of the real features.     

Multiple criteria linear programming model for portfolio selection

The portfolio selection problem is usually considered as a bicriteria optimization problem where a reasonable trade-off between expected rate of return and risk is sought. In the classical Markowitz model the risk is measured with variance, thus generating a quadratic programming model. The Markowitz model is frequently criticized as not consistent with axiomatic models of preferences for choice under risk. Models consistent with the preference axioms are based on the relation of stochastic dominance or on expected utility theory. The former is quite easy to implement for pairwise comparisons of given portfolios whereas it does not offer any computational tool to analyze the portfolio selection problem. The latter, when used for the portfolio selection problem, is restrictive in modeling preferences of investors. In this paper, a multiple criteria linear programming model of the portfolio selection problem is developed. The model is based on the preference axioms for choice under risk. Nevertheless, it allows one to employ the standard multiple criteria procedures to analyze the portfolio selection problem. It is shown that the classical mean-risk approaches resulting in linear programming models correspond to specific solution techniques applied to our multiple criteria model. This revised version was published online in June 2006 with corrections to the Cover Date.     

误差服从多元t分布的一类线性模型下参数估计的若干注记

研究一类线性模型下参数估计的若干问题．这类模型包含了多个因变量线性模型、增长曲线模型、扩充的增长曲线模型、似乎不相关回归方程组、方差分量模型等常用模型．在这类线性模型下,证明了当误差服从多元t分布时与误差服从多元正态分布时,具有相同的完全统计量和无偏估计,且在后一种情况下的充分统计量必为前一种情况下的充分统计量．对于带有多种协方差结构的前述几种模型,把在误差服从多元正态分布下,相应的协方差阵及有关参数的一致最小风险无偏(UMRU)估计存在性的结论推广到了相应的误差服从多元t分布情形．此外,对于误差服从多元t分布的这类统一的线性模型,给出了回归系数的线性可估函数的无偏估计的协方差阵的C-R下界．     

On estimation in multivariate linear calibration with elliptical errors

The estimation problem in multivariate linear calibration with elliptical errors is considered under a loss function which can be derived from the Kullback-Leibler distance. First, we discuss the problem under normal errors and give unbiased estimate of risk of an alternative estimator by means of the Stein and Stein-Haff identities for multivariate normal distribution. From the unbiased estimate of risk, it is shown that a shrinkage estimator improves on the classical estimator under the loss function. Furthermore, from the extended Stein and Stein-Haff identities for our elliptically contoured distribution, the above result under normal errors is extended to the estimation problem under elliptical errors. We show that the shrinkage estimator obtained under normal models is better than the classical estimator under elliptical errors with the above loss function and hence we establish the robustness of the above shrinkage estimator.