Admissible prediction in superpopulation models with random regression coefficients under matrix loss function

Admissible prediction problems in finite populations with arbitrary rank under matrix loss function are investigated. For the general random effects linear model, we obtained the necessary and sufficient conditions for a linear predictor of the linearly predictable variable to be admissible in the two classes of homogeneous linear predictors and all linear predictors and the class that contains all predictors, respectively. Moreover, we prove that the best linear unbiased predictors (BLUPs) of the population total and the finite population regression coefficient are admissible under different assumptions of superpopulation models respectively.     

An adjusted maximum likelihood method for solving small area estimation problems

For the well-known Fay-Herriot small area model, standard variance component estimation methods frequently produce zero estimates of the strictly positive model variance. As a consequence, an empirical best linear unbiased predictor of a small area mean, commonly used in small area estimation, could reduce to a simple regression estimator, which typically has an overshrinking problem. We propose an adjusted maximum likelihood estimator of the model variance that maximizes an adjusted likelihood defined as a product of the model variance and a standard likelihood (e.g., a profile or residual likelihood) function. The adjustment factor was suggested earlier by Carl Morris in the context of approximating a hierarchical Bayes solution where the hyperparameters, including the model variance, are assumed to follow a prior distribution. Interestingly, the proposed adjustment does not affect the mean squared error property of the model variance estimator or the corresponding empirical best linear unbiased predictors of the small area means in a higher order asymptotic sense. However, as demonstrated in our simulation study, the proposed adjustment has a considerable advantage in small sample inference, especially in estimating the shrinkage parameters and in constructing the parametric bootstrap prediction intervals of the small area means, which require the use of a strictly positive consistent model variance estimate.     

Robust Bayesian prediction and estimation under a squared log error loss function

Robust Bayesian analysis is concerned with the problem of making decisions about some future observation or an unknown parameter, when the prior distribution belongs to a class Γ instead of being specified exactly. In this paper, the problem of robust Bayesian prediction and estimation under a squared log error loss function is considered. We find the posterior regret Γ-minimax predictor and estimator in a general class of distributions. Furthermore, we construct the conditional Γ-minimax, most stable and least sensitive prediction and estimation in a gamma model. A prequential analysis is carried out by using a simulation study to compare these predictors.     

Measures of influence for the functional linear model with scalar response

This paper studies how to identify influential observations in the functional linear model in which the predictor is functional and the response is scalar. Measurement of the effects of a single observation on estimation and prediction when the model is estimated by the principal components method is undertaken. For that, three statistics are introduced for measuring the influence of each observation on estimation and prediction of the functional linear model with scalar response that are generalizations of the measures proposed for the standard regression model by [D.R. Cook, Detection of influential observations in linear regression, Technometrics 19 (1977) 15-18; D. Peña, A new statistic for influence in linear regression, Technometrics 47 (2005) 1-12] respectively. A smoothed bootstrap method is proposed to estimate the quantiles of the influence measures, which allows us to point out which observations have the larger influence on estimation and prediction. The behavior of the three statistics and the quantile estimation bootstrap based method is analyzed via a simulation study. Finally, the practical use of the proposed statistics is illustrated by the analysis of a real data example, which show that the proposed measures are useful for detecting heterogeneity in the functional linear model with scalar response.     

On same-realization prediction in an infinite-order autoregressive process

Let observations come from an infinite-order autoregressive (AR) process. For predicting the future of the observed time series (referred to as the same-realization prediction), we use the least-squares predictor obtained by fitting a finite-order AR model. We also allow the order to become infinite as the number of observations does in order to obtain a better approximation. Moment bounds for the inverse sample covariance matrix with an increasing dimension are established under various conditions. We then apply these results to obtain an asymptotic expression for the mean-squared prediction error of the least-squares predictor in same-realization and increasing-order settings. The second-order term of this expression is the sum of two terms which measure both the goodness of fit and model complexity. It forms the foundation for a companion paper by Ing and Wei (Order selection for same-realization predictions in autoregressive processes, Technical report C-00-09, Institute of Statistical Science, Academia Sinica, Taipei, Taiwan, ROC, 2000) which provides the first theoretical verification that AIC is asymptotically efficient for same-realization predictions. Finally, some comparisons between the least-squares predictor and the ridge regression predictor are also given.     

Conditionally specified models and dimension reduction in the exponential families

We consider informative dimension reduction for regression problems with random predictors. Based on the conditional specification of the model, we develop a methodology for replacing the predictors with a smaller number of functions of the predictors. We apply the method to the case where the inverse conditional model is in the linear exponential family. For such an inverse model and the usual Normal forward regression model it is shown that, for any number of predictors, the sufficient summary has dimension two or less. In addition, we develop a test of dimensionality. The relationship of our method with the existing dimension reduction theory based on the marginal distribution of the predictors is discussed.     

Uncertainty under a multivariate nested-error regression model with logarithmic transformation

This work aims to predict exponentials of mixed effects under a multivariate linear regression model with one random factor. Such quantities are of particular interest in prediction problems where the dependent variable is the logarithm of the variable that is the object of inference. Bias-corrected empirical predictors of the target quantities are defined. A second-order approximation for the mean crossed product error of two of these predictors is obtained, where the mean squared error is a particular case. An estimator of the mean crossed product error with second-order bias is proposed. Finally, results are illustrated through an application related to small area estimation.     

Nonparametric time series prediction: A semi-functional partial linear modeling

There is a recent interest in developing new statistical methods to predict time series by taking into account a continuous set of past values as predictors. In this functional time series prediction approach, we propose a functional version of the partial linear model that allows both to consider additional covariates and to use a continuous path in the past to predict future values of the process. The aim of this paper is to present this model, to construct some estimates and to look at their properties both from a theoretical point of view by means of asymptotic results and from a practical perspective by treating some real data sets. Although the literature on the use of parametric or nonparametric functional modeling is growing, as far as we know, this is the first paper on semiparametric functional modeling for the prediction of time series.     

Sufficient dimension reduction for the conditional mean with a categorical predictor in multivariate regression

Recent sufficient dimension reduction methodologies in multivariate regression do not have direct application to a categorical predictor. For this, we define the multivariate central partial mean subspace and propose two methodologies to estimate it. The first method uses the ordinary least squares. Chi-squared distributed statistics for dimension tests are constructed, and an estimate of the target subspace is consistent and efficient. Moreover, the effects of continuous predictors can be tested without assuming any model. The second method extends Iterative Hessian Transformation to this context. For dimension estimation, permutation tests are used. Simulated and real data examples for illustrating various properties of the proposed methods are presented.     

Optimal prediction for linear regression with infinitely many parameters

The problem of optimal prediction in the stochastic linear regression model with infinitely many parameters is considered. We suggest a prediction method that outperforms asymptotically the ordinary least squares predictor. Moreover, if the random errors are Gaussian, the method is asymptotically minimax over ellipsoids in ?₂. The method is based on a regularized least squares estimator with weights of the Pinsker filter. We also consider the case of dynamic linear regression, which is important in the context of transfer function modeling.     

Nonconcave penalized inverse regression in single-index models with high dimensional predictors

In this paper we aim to estimate the direction in general single-index models and to select important variables simultaneously when a diverging number of predictors are involved in regressions. Towards this end, we propose the nonconcave penalized inverse regression method. Specifically, the resulting estimation with the SCAD penalty enjoys an oracle property in semi-parametric models even when the dimension, p_n, of predictors goes to infinity. Under regularity conditions we also achieve the asymptotic normality when the dimension of predictor vector goes to infinity at the rate of p_n=o(n^1/3) where n is sample size, which enables us to construct confidence interval/region for the estimated index. The asymptotic results are augmented by simulations, and illustrated by analysis of an air pollution dataset.     

关于预测误差平方和最小准则的几点看法

在应用桃棣荣等提出的预测误差平方和（ＰＲＥＳＳ）最小准则逐步算法的过程中，发现单纯应用ＰＲＥＳＳ值来控制引进因子数目有时不很有效。因此建议：１）当引进因子过多时，采用赤池信息准则对因子数做进一步控制。２）当引进因子过少时，取消原算法中关于ＰＲＥＳＳ值上界的规定和增加候选因子数以增加引进因子数。最后，对应用ＰＲＥＳＳ准则时所选因子的优良性进行了讨论     

A sequential predictor retraining algorithm and its application to market prediction

In many applications, there are a large number of predictors, designed manually or trained automatically, to predict the same outcome. Much research has been devoted to the design of algorithms that can effectively select/combine these predictors to generate a more accurate ensemble predictor. The collaborative training algorithms from attribute distributed learning provide batch-processing solutions for scenarios in which the individual predictors are heterogeneous, taking different inputs and employing different models. However, in some applications, for example financial market prediction, it is desirable to use an online approach. In this paper, an innovative online algorithm is proposed, stemming from the collaborative training algorithms developed for attribute distributed learning. It sequentially takes new observations, simultaneously adjusting the way that the individual predictors are combined, and provides feedback to the individual predictors for them to be retrained in order to achieve a better ensemble predictor in real time. The efficacy of this new algorithm is demonstrated by extensive simulations on both artificial and real data, and particularly for financial market data. A trading strategy constructed from the ensemble predictor shows strong performance when applied to financial market prediction.     

矩阵损失下一般Gauss-Markov模型线性预测的可容许性

本文研究了一般Gauss-Markov模型中线性可预测变量的线性预测的可容许性.在给出线性预测可容许性定义的基础上,通过构造一个特殊的常量矩阵D0,分别得到了齐次和非齐次线性预测类中可容许的充要条件.     

Quadratic prediction and quadratic sufficiency in finite populations

The problem of quadratic prediction for population quadratic quantities in finite populations has been considered in the literature. In this paper, we mainly aim at extending the ordinary quadratic prediction problems to a general case, and derive the representations of the two essentially unique optimal predictors: one is an optimal invariant quadratic unbiased predictor, and the other is an optimal invariant quadratic (potentially) biased predictor. Further, we show that the two predictors are nonnegative and reasonable by considering an extreme situation, and apply resulting conclusions to a special model with a compound symmetric variance matrix. In addition, we propose a notion of quadratic sufficiency with regard to the optimal prediction problems by employing materials derived in the first part, and investigate corresponding characterizations in detail.     

Conditional and unconditional methods for selecting variables in linear mixed models

In the problem of selecting the explanatory variables in the linear mixed model, we address the derivation of the (unconditional or marginal) Akaike information criterion (AIC) and the conditional AIC (cAIC). The covariance matrices of the random effects and the error terms include unknown parameters like variance components, and the selection procedures proposed in the literature are limited to the cases where the parameters are known or partly unknown. In this paper, AIC and cAIC are extended to the situation where the parameters are completely unknown and they are estimated by the general consistent estimators including the maximum likelihood (ML), the restricted maximum likelihood (REML) and other unbiased estimators. We derive, related to AIC and cAIC, the marginal and the conditional prediction error criteria which select superior models in light of minimizing the prediction errors relative to quadratic loss functions. Finally, numerical performances of the proposed selection procedures are investigated through simulation studies.     

Variable selection in model-based discriminant analysis

A general methodology for selecting predictors for Gaussian generative classification models is presented. The problem is regarded as a model selection problem. Three different roles for each possible predictor are considered: a variable can be a relevant classification predictor or not, and the irrelevant classification variables can be linearly dependent on a part of the relevant predictors or independent variables. This variable selection model was inspired by a previous work on variable selection in model-based clustering. A BIC-like model selection criterion is proposed. It is optimized through two embedded forward stepwise variable selection algorithms for classification and linear regression. The model identifiability and the consistency of the variable selection criterion are proved. Numerical experiments on simulated and real data sets illustrate the interest of this variable selection methodology. In particular, it is shown that this well ground variable selection model can be of great interest to improve the classification performance of the quadratic discriminant analysis in a high dimension context.     

Linex Unbiasedness in a Prediction Problem

A statistical prediction problem under LINEX loss function is considered. Some results about LINEX-unbiased predictor are derived and the best LINEX-unbiased predictor is given. We also show that the best risk-unbiased predictor is equal to the best equivariant predictor in the location family.     

On Stein’s lemma, dependent covariates and functional monotonicity in multi-dimensional modeling

Tracking the correct directions of monotonicity in multi-dimensional modeling plays an important role in interpreting functional associations. In the presence of multiple predictors, we provide empirical evidence that the observed monotone directions via parametric, nonparametric or semiparametric fit of commonly used multi-dimensional models may entirely violate the actual directions of monotonicity. This breakdown is caused primarily by the dependence structure of covariates, with negligible influence from the bias of function estimation. To examine the linkage between the dependent covariates and monotone directions, we first generalize Stein’s Lemma for random variables which are mutually independent Gaussian to two important cases: dependent Gaussian, and independent non-Gaussian. We show that in both two cases, there is an explicit one-to-one correspondence between the monotone directions of a multi-dimensional function and the signs of a deterministic surrogate vector. Moreover, we demonstrate that the second case can be extended to accommodate a class of dependent covariates. This generalization further enables us to develop a de-correlation transform for arbitrarily dependent covariates. The transformed covariates preserve modeling interpretability with little loss in modeling efficiency. The simplicity and effectiveness of the proposed method are illustrated via simulation studies and real data application.