Model selection for regression on a fixed design

We deal with the problem of estimating some unknown regression function involved in a regression framework with deterministic design points. For this end, we consider some collection of finite dimensional linear spaces (models) and the least-squares estimator built on a data driven selected model among this collection. This data driven choice is performed via the minimization of some penalized model selection criterion that generalizes on Mallows' C _p . We provide non asymptotic risk bounds for the so-defined estimator from which we deduce adaptivity properties. Our results hold under mild moment conditions on the errors. The statement and the use of a new moment inequality for empirical processes is at the heart of the techniques involved in our approach. Received: 2 July 1997 / Revised version: 20 September 1999 / Published online: 6 July 2000     

Unbiased Recursive Partitioning: A Conditional Inference Framework

Recursive binary partitioning is a popular tool for regression analysis. Two fundamental problems of exhaustive search procedures usually applied to fit such models have been known for a long time: overfitting and a selection bias towards covariates with many possible splits or missing values. While pruning procedures are able to solve the overfitting problem, the variable selection bias still seriously affects the interpretability of tree-structured regression models. For some special cases unbiased procedures have been suggested, however lacking a common theoretical foundation. We propose a unified framework for recursive partitioning which embeds tree-structured regression models into a well defined theory of conditional inference procedures. Stopping criteria based on multiple test procedures are implemented and it is shown that the predictive performance of the resulting trees is as good as the performance of established exhaustive search procedures. It turns out that the partitions and therefore the models induced by both approaches are structurally different, confirming the need for an unbiased variable selection. Moreover, it is shown that the prediction accuracy of trees with early stopping is equivalent to the prediction accuracy of pruned trees with unbiased variable selection. The methodology presented here is applicable to all kinds of regression problems, including nominal, ordinal, numeric, censored as well as multivariate response variables and arbitrary measurement scales of the covariates. Data from studies on glaucoma classification, node positive breast cancer survival and mammography experience are re-analyzed.     

影响系统主因素综合筛选技术研究

系统影响因素过多不仅会增加系统建模难度,而且会增加系统模型的不确定性和数据收集工作量,因此提出一种影响系统的主因素综合筛选技术.技术将综合灰色关联度、普通相关技术、逐步回归方法、DEMATEL法以及系统动力学等方法的优点和缺点,通过彼此互补和相互融合技术,使得该技术能够有效地筛选出影响系统的主因素.通过在筛选影响船舶柴油机寿命周期费用主因素的应用,和其他方法的比较和分析,验证了本技术的可行性、有效性和先进性.     

Data Adaptive Ridging in Local Polynomial Regression

Abstract

When estimating a regression function or its derivatives, local polynomials are an attractive choice due to their flexibility and asymptotic performance. Seifert and Gasser proposed ridging of local polynomials to overcome problems with variance for random design while retaining their advantages. In this article we present a data-independent rule of thumb and a data-adaptive spatial choice of the ridge parameter in local linear regression. In a framework of penalized local least squares regression, the methods are generalized to higher order polynomials, to estimation of derivatives, and to multivariate designs. The main message is that ridging is a powerful tool for improving the performance of local polynomials. A rule of thumb offers drastic improvements; data-adaptive ridging brings further but modest gains in mean square error.     

Adaptive Free-Knot Splines

This article proposes a function estimation procedure using free-knot splines as well as an associated algorithm for implementation in nonparametric regression. In contrast to conventional splines with knots confined to distinct design points, the splines allow selection of knot numbers and replacement of knots at any location and repeated knots at the same location. This exibility leads to an adaptive spline estimator that adapts any function with inhomogeneous smoothness, including discontinuity, which substantially improves the representation power of splines. Due to uses of a large class of spline functions, knot selection becomes extremely important. The existing knot selection schemes—such as stepwise selection—suffer the difficulty of knot confounding and are unsuitable for our purpose. A new knot selection scheme is proposed using an evolutionary Monte Carlo algorithm and an adaptive model selection criterion. The evolutionary algorithm locates the optimal knots accurately, whereas the adaptive model selection strategy guards against the selection error in searching through a large candidate knot space. The performance of the procedure is examined and illustrated via simulations. The procedure provides a significant improvement in performance over the other competing adaptive methods proposed in the literature. Finally, usefulness of the procedure is illustrated by an application to actual dataset.     

Ein dichteunabhängiges Neutronenfeuchtemeßgerät

Voruntersuchungen ergaben, daß sich der Dichteeinfluß bei der Neutronenfeuchtemessung kompensieren läßt. Eine Meßanordnung muß dazu zwei Zählraten liefern, die jede für sich linear nur vom Wassergehalt je Volumeneinheit bzw. von der Dichte abhängt. Mit Hilfe eines Datenverarbeitungsgerätes sind dann Angaben der Feuchte in Gewichtsprozenten in bestimmten Zeitabständen möglich. Es wurden ein kombinierter Feuchte-Dichte-Meßkopf und ein Datenverarbeitungsgerät aufgebaut und an Modellsubstanzen erprobt. Es konnte gezeigt werden, daß die Meßwerte weitgehend dichteunabhängig sind. Allerdings sind häufige Eichkontrollen notwendig.     

Monotonically Overrelaxed EM Algorithms

We explore the idea of overrelaxation for accelerating the expectation-maximization (EM) algorithm, focusing on preserving its simplicity and monotonic convergence properties. It is shown that in many cases, a trivial modification in the M-step results in an algorithm that maintains monotonic increase in the log-likelihood, but can have an appreciably faster convergence rate, especially when EM is very slow. The method is applicable to more general fixed point algorithms. Its simplicity and effectiveness are illustrated with several statistical problems, including probit regression, least absolute deviations regression, Poisson inverse problems, and finite mixtures. This article has supplemental materials available online.     

Characterizing nonconstant instrumental variance in emerging miniaturized analytical techniques

Measurement variance is a crucial aspect of quantitative chemical analysis. Variance directly affects important analytical figures of merit, including detection limit, quantitation limit, and confidence intervals. Most reported analyses for emerging analytical techniques implicitly assume constant variance (homoskedasticity) by using unweighted regression calibrations. Despite the assumption of constant variance, it is known that most instruments exhibit heteroskedasticity, where variance changes with signal intensity. Ignoring nonconstant variance results in suboptimal calibrations, invalid uncertainty estimates, and incorrect detection limits. Three techniques where homoskedasticity is often assumed were covered in this work to evaluate if heteroskedasticity had a significant quantitative impact—naked-eye, distance-based detection using paper-based analytical devices (PADs), cathodic stripping voltammetry (CSV) with disposable carbon-ink electrode devices, and microchip electrophoresis (MCE) with conductivity detection. Despite these techniques representing a wide range of chemistries and precision, heteroskedastic behavior was confirmed for each. The general variance forms were analyzed, and recommendations for accounting for nonconstant variance discussed. Monte Carlo simulations of instrument responses were performed to quantify the benefits of weighted regression, and the sensitivity to uncertainty in the variance function was tested. Results show that heteroskedasticity should be considered during development of new techniques; even moderate uncertainty (30%) in the variance function still results in weighted regression outperforming unweighted regressions. We recommend utilizing the power model of variance because it is easy to apply, requires little additional experimentation, and produces higher-precision results and more reliable uncertainty estimates than assuming homoskedasticity.     

基于粗糙回归的股票指数变化趋势研究

运用粗糙集结合回归分析预测上证指数变化趋势,首先对预测日前1个月、前1星期、前1天等8个不同时间段进行回归分析,然后根据获得的趋势建立决策表,然后运用粗糙集获取决策规则,最后利用得到的规则预测上证指数的变化趋势.测试结果表明,该方法准确率良好.