Prediction error criterion for selecting variables in a linear regression model

Several criteria, such as CV, C _p , AIC, CAIC, and MAIC, are used for selecting variables in linear regression models. It might be noted that C _p has been proposed as an estimator of the expected standardized prediction error, although the target risk function of CV might be regarded as the expected prediction error R _PE. On the other hand, the target risk function of AIC, CAIC, and MAIC is the expected log-predictive likelihood. In this paper, we propose a prediction error criterion, PE, which is an estimator of the expected prediction error R _PE. Consequently, it is also a competitor of CV. Results of this study show that PE is an unbiased estimator when the true model is contained in the full model. The property is shown without the assumption of normality. In fact, PE is demonstrated as more faithful for its risk function than CV. The prediction error criterion PE is extended to the multivariate case. Furthermore, using simulations, we examine some peculiarities of all these criteria.     

Order Determination for Multivariate Autoregressive Processes Using Resampling Methods

LetX₁, …, X_nbe observations from a multivariate AR(p) model with unknown orderp. A resampling procedure is proposed for estimating the orderp. The classical criteria, such as AIC and BIC, estimate the orderpas the minimizer of the function[formula]wherenis the sample size,kis the order of the fitted model, Σ²_kis an estimate of the white noise covariance matrix, andC_nis a sequence of specified constants (for AIC,C_n=2m²/n, for Hannan and Quinn's modification of BIC,C_n=2m²(ln ln n)/n, wheremis the dimension of the data vector). A resampling scheme is proposed to estimate an improved penalty factorC_n. Conditional on the data, this procedure produces a consistent estimate ofp. Simulation results support the effectiveness of this procedure when compared with some of the traditional order selection criteria. Comments are also made on the use of Yule–Walker as opposed to conditional least squares estimations for order selection.     

On monomialk-Buchsbaum curves inP r

This paper is deveted to the study of projective monomialk-Buchsbaum curves C. First, using the theory of affine semigroup rings, we give a criterion forC to bek-Buchsbaum. Then we give some lower and upper bounds for the numberk _c such thatC is strictlyk _c-Buchsbaum. For some classes of monomial curves we can computek _C explicity.     

Multi-sample cluster analysis using Akaike's Information Criterion

Summary Multi-sample cluster analysis, the problem of grouping samples, is studied from an information-theoretic viewpoint via Akaike's Information Criterion (AIC). This criterion combines the maximum value of the likelihood with the number of parameters used in achieving that value. The multi-sample cluster problem is defined, and AIC is developed for this problem. The form of AIC is derived in both the multivariate analysis of variance (MANOVA) model and in the multivariate model with varying mean vectors and variance-covariance matrices. Numerical examples are presented for AIC and another criterion calledw-square. The results demonstrate the utility of AIC in identifying the best clustering alternatives. This research was supported by Office of Naval Research Contract N00014-80-C-0408, Task NR042-443 and Army Research Office Contract DAAG 29-82-K-0155, at the University of Illinois at Chicago.     

Selection of smoothing parameters in<Emphasis Type="Italic">B</Emphasis>-spline nonparametric regression models using information criteria

We consider the use ofB-spline nonparametric regression models estimated by the maximum penalized likelihood method for extracting information from data with complex nonlinear structure. Crucial points inB-spline smoothing are the choices of a smoothing parameter and the number of basis functions, for which several selectors have been proposed based on cross-validation and Akaike information criterion known as AIC. It might be however noticed that AIC is a criterion for evaluating models estimated by the maximum likelihood method, and it was derived under the assumption that the ture distribution belongs to the specified parametric model. In this paper we derive information criteria for evaluatingB-spline nonparametric regression models estimated by the maximum penalized likelihood method in the context of generalized linear models under model misspecification. We use Monte Carlo experiments and real data examples to examine the properties of our criteria including various selectors proposed previously.     

A criterion for the affine equivalence of cell complexes inR d and convex polyhedra inR d+1

A criterion is given that decides, for a convex tilingC ofR ^d , whetherC is the projection of the faces in the boundary of some convex polyhedronP inR ^d+1. Its applicability is restricted neither by the properties nor by the dimension ofC. It turns out to be conceptually simpler than other criteria and allows the easy examination of various classes of cell complexes. In addition, the criterion is constructive, that is, it can be used to constructP provided it exists.Research was supported by the Austrian Fonds zur Foerderung der wissenschaftlichen Forschung.     

Characterizations of classes of risk measures by dispersive orders

In this paper, a class C₁ of risk measures, which generalizes the class of risk measures for the right-tail deviation suggested by Wang [Wang, S., 1998. An actuarial index of the right-tail risk. North Amer. Actuarial J. 2, 88–101], is characterized in terms of dispersive order. If dispersive order does not hold, unanimous comparisons are still possible by restricting our attention to a subclass C₂C₁ and then the criterion is the excess-wealth order. Sufficient conditions for stochastic equivalence of excess-wealth ordered random variables are derived in terms of some particular measures of C₂.     

On robust cross-validation for nonparametric smoothing

An essential problem in nonparametric smoothing of noisy data is a proper choice of the bandwidth or window width, which depends on a smoothing parameter $k$ . One way to choose $k$ based on the data is leave-one-out-cross-validation. The selection of the cross-validation criterion is similarly important as the choice of the smoother. Especially, when outliers are present, robust cross-validation criteria are needed. So far little is known about the behaviour of robust cross-validated smoothers in the presence of discontinuities in the regression function. We combine different smoothing procedures based on local constant fits with each of several cross-validation criteria. These combinations are compared in a simulation study under a broad variety of data situations with outliers and abrupt jumps. There is not a single overall best cross-validation criterion, but we find Boente-cross-validation to perform well in case of large percentages of outliers and the Tukey-criterion in case of data situations with jumps, even if the data are contaminated with outliers.     

On L1 and Chebyshev estimation

The problem considered here is that of fitting a linear function to a set of points. The criterion normally used for this is least squares. We consider two alternatives, viz., least sum of absolute deviations (called the L₁ criterion) and the least maximum absolute deviation (called the Chebyshev criterion). Each of these criteria give rise to a linear program. We develop some theoretical properties of the solutions and in the light of these, examine the suitability of these criteria for linear estimation. Some of the estimates obtained by using them are shown to be counter-intuitive.     

A Computational Study of Shifting Bottleneck Procedures for Shop Scheduling Problems

We examine the performance of Shifting Bottleneck (SB) heuristics for shop scheduling problems where the performance measure to be minimized is makespan (C _max) or maximum lateness (L _max). Extensive computational experiments are conducted on benchmark problems from the literature as well as several thousand randomly generated test problems with three different routing structures and up to 1000 operations. Several different versions of SB are examined to determine the effect on solution quality and time of different subproblem solution procedures, reoptimization procedures and bottleneck selection criteria. Results show that the performance of SB is significantly affected by job routings, and that SB with optimal subproblem solutions and full reoptimization at each iteration consistently outperforms dispatching rules, but requires high computation times for large problems. High quality subproblem solutions and reoptimization procedures are essential to obtaining good solutions. We also show that schedules developed by SB to minimize L _max perform well with respect to several other performance measures, rendering them more attractive for practical use.     

On the efficiency of selection criteria in spline regression

This paper concerns the cubic smoothing spline approach to nonparametric regression. After first deriving sharp asymptotic formulas for the eigenvalues of the smoothing matrix, the paper uses these formulas to investigate the efficiency of different selection criteria for choosing the smoothing parameter. Special attention is paid to the generalized maximum likelihood (GML), C _p and extended exponential (EE) criteria and their marginal Bayesian interpretation. It is shown that (a) when the Bayesian model that motivates GML is true, using C _p to estimate the smoothing parameter would result in a loss of efficiency with a factor of 10/3, proving and strengthening a conjecture proposed in Stein (1990); (b) when the data indeed come from the C _p density, using GML would result in a loss of efficiency of ; (c) the loss of efficiency of the EE criterion is at most 1.543 when the data are sampled from its consistent density family. The paper not only studies equally spaced observations (the setting of Stein, 1990), but also investigates general sampling scheme of the design points, and shows that the efficiency results remain the same in both cases.This work is supported in part by NSF grant DMS-0204674 and Harvard University Clark-Cooke Fund. Mathematics Subject Classification (2000):Primary: 62G08; Secondary: 62G20     

Investigations of interlaminar fracture toughness of laminated polymeric composites

The behavior of interlaminar fracture of fiber reinforced laminated polymeric composites has been investigated in modes I, II, and different mixed mode I/II ratios. The experimental investigations were carried out by using conventional beam specimens and the compound version of the CTS (compact tension shear) specimen. In this study, a compound version of the CTS specimen is used for the first time to determine the interlaminar fracture toughness of composites. In order to verify the results obtained by the CTS tests, conventional beam tests were also carried out. In the beam tests, specimens of double cantilever beam (DCB) and end notched flexure (ENF) were used to obtain the critical rates of the energy release for failure modes I and II. The CTS specimen is used to obtain different mixed mode ratios, from pure mode I to pure mode II, by varying the loading conditions. The highest mixed mode ratio obtained in the experiment was G _I /G _II =60. The data obtained from these tests were analyzed by the finite element method. The separated critical rates G _I and G _II of the energy release were calculated by using the modified virtual crack closure integral (MVCCI) method. The experimental investigations were performed on a unidirectional glass/epoxy composite. The results obtained by the beam and CTS tests were compared. It was found that the interlaminar fracture toughness G _IC ^init of mode I at crack initiation and the corresponding value G _II ^Cinit of mode II obtained by the conventional beam and the CTS tests were in rather good agreement. The experimental results of interlaminar fracture of mixed mode were used to obtain the parameters required for the failure criterion. The two different failure criteria were compared. The best correlation with the experimental data was obtained by using the failure criterion proposed by Wu in 1967 containing linear and quadratic terms of the rates of the energy release.Presented at the 10th International Conference on the Mechanics of Composite Materials (Riga, April 20–23, 1998).Translated from Mekhanika Kompozitnykh Materialov, Vol. 34, No. 3, pp. 307–322, May–June, 1998.     

Multimodel inference based on smoothed information criteria

The multimodel inference makes statistical inferences from a set of plausible models rather than from a single model. In this paper, we focus on the multimodel inference based on smoothed information criteria proposed by seminal monographs(see Buckland et al.(1997) and Burnham and Anderson(2003)), which are termed as smoothed Akaike information criterion(SAIC) and smoothed Bayesian information criterion(SBIC)methods. Due to their simplicity and applicability, these methods are very widely used in many fields. By using an illustrative example and deriving limiting properties for the weights in the linear regression, we find that the existing variance estimation for SAIC is not applicable because of a restrictive condition, but for SBIC it is applicable. Especially, we propose a simulation-based inference for SAIC based on the limiting properties. Both the simulation study and the real data example show the promising performance of the proposed simulationbased inference.     

Optimal Criterion Weights in Repetitive Multicriteria Decision-Making

In repetitive judgmental discrete decision-making with multiple criteria, the decision maker usually behaves as if there is a set of appropriate criterion weights such that the decisions chosen are based on the weighted sum of all the criteria. Many different procedures for estimating these implied criterion weights have been proposed. Most of these procedures emphasize the preference trade-off among the multiple criteria of the decision maker, and thus the criterion weights obtained are not directly related to the hit ratio of matching decisions. Based on past data, statistical discriminant analysis can be used to determine the implied criterion weights that would reflect the past decisions. The most interesting performance measure is the hit ratio. In this work, we use the integer linear goal-programming technique to determine optimal criterion weights which minimize the number of misclassification of decisions. The linear goal-programming formulation has m constraints and m + k + 1 variables, where m is the number of cases and k is the number of criteria. Empirical study is done by using two different procedures on the actual past admission data of an M.B.A. programme. The hit ratios of the different procedures are compared.     

Conditional and unconditional methods for selecting variables in linear mixed models

In the problem of selecting the explanatory variables in the linear mixed model, we address the derivation of the (unconditional or marginal) Akaike information criterion (AIC) and the conditional AIC (cAIC). The covariance matrices of the random effects and the error terms include unknown parameters like variance components, and the selection procedures proposed in the literature are limited to the cases where the parameters are known or partly unknown. In this paper, AIC and cAIC are extended to the situation where the parameters are completely unknown and they are estimated by the general consistent estimators including the maximum likelihood (ML), the restricted maximum likelihood (REML) and other unbiased estimators. We derive, related to AIC and cAIC, the marginal and the conditional prediction error criteria which select superior models in light of minimizing the prediction errors relative to quadratic loss functions. Finally, numerical performances of the proposed selection procedures are investigated through simulation studies.     

Regular reflection in potential flow and the sonic criterion

We study the classical problem of self-similar reflection of shocks at a ramp, modeled by potential flow with γ-law pressure. Depending on corner angle θ and upstream Mach number M_I , either regular (RR) or Mach reflections occur. There are several conflicting transition criteria predicting the corner angle at which the type of reflection changes. We show that in some cases, in particular M_I = 1 and γ = 5/3, an exact RR solution exists for all θ specified by the sonic criterion. Thus all weaker criteria are false. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

An Application of Adaptive Independent Chain Metropolis–Hastings Algorithms in Bayesian Hazard Rate Estimation

In this paper, we consider adaptive independent chain (AIC) Metropolis–Hastings algorithms as introduced in a special context in Gåsemyr et al. (2001) and developed theoretically in Gåsemyr (2003). The algorithms aim at producing samples from a specific target distribution , and are adaptive, non-Markovian versions of the Metropolis–Hastings independent chain. A certain parametric class of possible proposal distributions is fixed, and the parameters of the proposal distribution are updated periodically on the basis of the recent history of the chain, thereby obtaining proposals that get ever closer to . In the former paper a version of these algorithms was shown to be very efficient compared to standard simulation techniques when applied to Bayesian inference in reliability models with at most three dependent parameters. The aim of the present paper is to investigate the performance of the AIC algorithm when the number of dependent parameters and the complexity of the model increases. As a test case we consider a model treated in Arjas and Gasbarra (1996). The target distribution is the posterior distribution for the vector X=(X ₁,...,X _n ) of dependent parameters, representing a piecewise constant approximation to the hazard rate X(t), where t ₀ t t _n . Especially, for the case n=12 it turned out that some versions of the AIC were very efficient compared to standard simulation techniques and also to the algorithm applied in Arjas and Gasbarra (1996). This includes a version of the componentwise adaptive independent chain the basic idea of which was given in Gåsemyr (2003).     

A multistage algorithm for best-subset model selection based on the Kullback–Leibler discrepancy

The selection of a best-subset regression model from a candidate family is a common problem that arises in many analyses. The Akaike information criterion (AIC) and the corrected AIC (\(\text {AIC}_c\)) are frequently used for this purpose. AIC and \(\text {AIC}_c\) are designed to estimate the expected Kullback–Leibler discrepancy. For best-subset selection, both AIC and \(\text {AIC}_c\) are negatively biased, and the use of either criterion will lead to the selection of overfitted models. To correct for this bias, we introduce an “improved” AIC variant, \(\text {AIC}_i\), which has a penalty term evaluated using Monte Carlo simulation. A multistage model selection procedure \(\text {AIC}_{\text {aps}}\), which utilizes \(\text {AIC}_i\), is proposed for best-subset selection. Simulation studies are compiled to compare the performances of the different model selection methods.     

A Dual Approach to Constrained Interpolationfrom a Convex Subset of Hilbert Space

Many interesting and important problems of best approximationare included in (or can be reduced to) one of the followingtype: in a Hilbert spaceX, find the best approximationP_K(x) to anyxXfrom the setKC∩A⁻¹(b),whereCis a closed convex subset ofX,Ais a bounded linearoperator fromXinto a finite-dimensional Hilbert spaceY, andbY. The main point of this paper is to show thatP_K(x)isidenticaltoP_C(x+A*y)—the best approximationto a certain perturbationx+A*yofx—from the convexsetCor from a certain convex extremal subsetC_bofC. Thelatter best approximation is generally much easier to computethan the former. Prior to this, the result had been known onlyin the case of a convex cone or forspecialdata sets associatedwith a closed convex set. In fact, we give anintrinsic characterizationof those pairs of setsCandA⁻¹(b) for which this canalways be done. Finally, in many cases, the best approximationP_C(x+A*y) can be obtained numerically from existingalgorithms or from modifications to existing algorithms. Wegive such an algorithm and prove its convergence     

CONTINUITY OF THE ONE-SIDED BEST UNIFORM APPROXIMATION OPERATOR

Abstract

The purpose of this paper is to relate the continuity and selection properties of the one-sided best uniform approximation operator to similar properties of the metric projection. Let M be a closed subspace of C(T) which contains constants. Then the one-sided best uniform approximation operator is Hausdorff continuous (resp. Lipschitz continuous) on C(T) if and only if the metric projection P_M is Haudorff continuous (resp. Lipschitz continuous) on C(T). Also, the metric projection P_M admits a continuous (resp. Lipschitz continuous) selection if and only if the one-sided best uniform approximation operator admits a continuous (resp. Lipschitz continuous) selection.