AIC, Overfitting Principles, and the Boundedness of Moments of Inverse Matrices for Vector Autotregressions and Related Models

In his somewhat informal derivation, Akaike (in “Proceedings of the 2nd International Symposium Information Theory” (C. B. Petrov and F. Csaki, Eds.), pp. 610–624, Academici Kiado, Budapest, 1973) obtained AIC's parameter-count adjustment to the log-likelihood as a bias correction: it yields an asymptotically unbiased estimate of the quantity that measures the average fit of the estimated model to an independent replicate of the data used for estimation. We present the first mathematically complete derivation of an analogous property of AIC for comparing vector autoregressions fit to weakly stationary series. As a preparatory result, we derive a very general “overfitting principle,” first formulated in a more limited context in Findley (Ann. Inst. Statist. Math.43, 509–514, 1991), asserting that a natural measure of an estimated model's overfit due to parameter estimation is equal, asymptotically, to a measure of its accuracy loss with independent replicates. A formal principle of parsimony for fitted models is obtained from this, which for nested models, covers the situation in which all models considered are misspecified. To prove these results, we establish a set of general conditions under which, for each τ1, the absolute τth moments of the entries of the inverse matrices associated with least squares estimation are bounded for sufficiently large sample sizes.