Improved customer choice predictions using ensemble methods

In this paper various ensemble learning methods from machine learning and statistics are considered and applied to the customer choice modeling problem. The application of ensemble learning usually improves the prediction quality of flexible models like decision trees and thus leads to improved predictions. We give experimental results for two real-life marketing datasets using decision trees, ensemble versions of decision trees and the logistic regression model, which is a standard approach for this problem. The ensemble models are found to improve upon individual decision trees and outperform logistic regression.     

决策树分类模型在工程项目评标风险预警中的应用

本文将数据挖掘中的决策树分类方法运用到工程项目评标数据分析,从200多个天津市工程项目招投标打分数据中,随机抽取15个招投标项目中的67个承包商的评标专家打分数据进行分析,得到中标承包商技术和商务评分分界点,进而得到工程项目潜在风险的预警阈值,然后借助因子分析辨识出风险来源并进行预警。     

Acoustic detection and classification of river boats

We present a robust algorithm to detect the arrival of a boat of a certain type when other background noises are present. It is done via the analysis of its acoustic signature against an existing database of recorded and processed acoustic signals. We characterize the signals by the distribution of their energies among blocks of wavelet packet coefficients. To derive the acoustic signature of the boat of interest, we use the Best Discriminant Basis method. The decision is made by combining the answers from the Linear Discriminant Analysis (LDA) classifier and from the Classification and Regression Trees (CART) that is also accompanied with an additional unit, called Aisles, that reduces false alarms rate. The proposed algorithm is a generic solution for process control that is based on a learning phase (training) followed by an automatic real time detection while minimizing the false alarms rate.     

A novel kernel Fisher discriminant analysis: constructing informative kernel by decision tree ensemble for metabolomics data analysis

Large amounts of data from high-throughput metabolomics experiments become commonly more and more complex, which brings an enormous amount of challenges to existing statistical modeling. Thus there is a need to develop statistically efficient approach for mining the underlying metabolite information contained by metabolomics data under investigation. In the work, we developed a novel kernel Fisher discriminant analysis (KFDA) algorithm by constructing an informative kernel based on decision tree ensemble. The constructed kernel can effectively encode the similarities of metabolomics samples between informative metabolites/biomarkers in specific parts of the measurement space. Simultaneously, informative metabolites or potential biomarkers can be successfully discovered by variable importance ranking in the process of building kernel. Moreover, KFDA can also deal with nonlinear relationship in the metabolomics data by such a kernel to some extent. Finally, two real metabolomics datasets together with a simulated data were used to demonstrate the performance of the proposed approach through the comparison of different approaches.     

Research on estimation model of the battery state of charge in a hybrid electric vehicle based on the classification and regression tree

ABSTRACT

In order to achieve the accurate estimation of state of charge (SOC) of the battery in a hybrid electric vehicle (HEV), this paper proposed a new estimation model based on the classification and regression tree (CART) which belongs to a kind of decision tree. The basic principle and modelling process of the CART decision tree were introduced in detail in this paper, and we used the voltage, current, and temperature of the battery in an HEV to estimate the value of SOC under the driving cycle. Meanwhile, we took the energy feedback of the HEV under the regenerative braking into consideration. The simulation data and experimental data were used to test the effectiveness of the estimation model of CART, and the results indicate that the proposed estimation model has high accuracy, the relative error of simulation is within 0.035, while the relative error of experiment is less than 0.05.     

落叶松受雅氏落叶松尺蠖危害程度光谱检测

近年来蒙古国雅氏落叶松尺蠖灾害不断加剧，逐渐逼近大兴安岭地区，将威胁我国北方森林生态系统安全。以现代遥感监测方法替代传统检测方法，及早掌握该虫害发生发展规律对防控有重要意义。为快速、大范围遥感监测雅氏落叶松尺蠖灾害，利用光谱分析技术研究了该害虫危害下落叶松受害程度检测模型。通过实测健康和轻度、中度、重度受害落叶松光谱，计算与比较不同受害程度落叶松原始光谱和去除包络线光谱的敏感度，揭示光谱敏感波段及去除包络线光谱敏感性。然后对去除包络线光谱进行一阶导数变换获得光谱特征参数并分析其随受害程度的变化特征，构建基于CART(分类与回归树)算法的落叶松受害程度光谱检测模型。研究表明：去除包络线光谱敏感性比原始光谱更显著，尤其在480～520 nm(蓝边)、640～720 nm(红谷、红边)、1 416～1 500 nm(短波红外谷)等波段内光谱敏感度介于0.1～2.0，而且出现了敏感峰现象。随受害程度增加，去除包络线光谱敏感性增强趋势比原始光谱更明显；在蓝边波段上去除包络线光谱敏感峰位置向短波方向移动，即502 nm→490 nm，而在红谷及红边、短波红外谷等波段上光谱敏感峰位置向长波方向移动，即664 nm→672 nm和1 436 nm→1 448 nm；红谷位置和短波红外谷位置归一化反射率以及红谷和短波红外谷面积呈上升趋势。在蓝边与红边波段内去除包络线光谱一阶导数对受害程度有明显响应，出现了波峰现象。随害虫危害程度加剧红边位置蓝移(718 nm→700 nm)，红边斜率及面积和蓝边斜率及面积呈下降趋势。基于此，利用红边斜率、红谷位置和短波红外谷位置归一化反射率、红谷和短波红外谷面积、蓝边斜率及面积等去除包络线光谱特征参数构建的CART模型对落叶松受害程度有很好的检测能力。与多元线性回归模型相比，CART模型检测精度更高，其Kappa系数达0.875。研究结果对雅氏落叶松尺蠖灾害的防治有参考价值。     

A comparison of statistical methods for prenatal screening for Down syndrome

Currently, prenatal screening for Down Syndrome (DS) uses the mother's age as well as three biochemical markers for risk prediction. Risk calculations for the biochemical markers use a quadratic discriminant function. In this paper we compare several classification procedures to quadratic discrimination methods for biochemical-based DS risk prediction, based on data from a prospective multicentre prenatal screening study. We investigate alternative methods including linear discriminant methods, logistic regression methods, neural network methods, and classification and regression-tree methods. Several experiments are performed, and in each experiment resampling methods are used to create training and testing data sets. The procedures on the test data set are summarized by the area under their receiver operating characteristic curves. In each experiment this process is repeated 500 times and then the classification procedures are compared. We find that several methods are superior to the currently used quadratic discriminant method for risk estimation for these data. The implications of these results for prenatal screening programs are discussed.     

Treed Regression

Abstract

Given a data set consisting of n observations on p independent variables and a single dependent variable, treed regression creates a binary tree with a simple linear regression function at each of the leaves. Each node of the tree consists of an inequality condition on one of the independent variables. The tree is generated from the training data by a recursive partitioning algorithm. Treed regression models are more parsimonious than CART models because there are fewer splits. Additionally, monotonicity in some or all of the variables can be imposed.     

Algorithms for manipulation of level sets of nonparametric density estimates

Summary  We present algorithms for finding the level set tree of a multivariate density estimate. That is, we find the separated components of level sets of the estimate for a series of levels, gather information on the separated components, such as volume and barycenter, and present the information together with the tree structure of the separated components. The algorithm proceeds by first building a binary tree which partitions the support of the density estimate, followed by bottom-up travels of this tree during which we join those parts of the level sets which touch each other. As a byproduct we present an algorithm for evaluating a kernel estimate on a large multidimensional grid. Since we find the barycenters of the separated components of the level sets also for high levels, our method finds the locations of local extremes of the estimate. Writing of this article was financed by Deutsche Forschungsgemeinschaft under project MA1026/8-1.     

Regression Tree Analysis Using TARGET

Regression trees are a popular alternative to classical regression methods. A number of approaches exist for constructing regression trees. Most of these techniques, including CART, are sequential in nature and locally optimal at each node split, so the final tree solution found may not be the best tree overall. In addition, small changes in the training data often lead to large changes in the final result due to the relative instability of these greedy tree-growing algorithms. Ensemble techniques, such as random forests, attempt to take advantage of this instability by growing a forest of trees from the data and averaging their predictions. The predictive performance is improved, but the simplicity of a single-tree solution is lost.

In earlier work, we introduced the Tree Analysis with Randomly Generated and Evolved Trees (TARGET) method for constructing classification trees via genetic algorithms. In this article, we extend the TARGET approach to regression trees. Simulated data and real world data are used to illustrate the TARGET process and compare its performance to CART, Bayesian CART, and random forests. The empirical results indicate that TARGET regression trees have better predictive performance than recursive partitioning methods, such as CART, and single-tree stochastic search methods, such as Bayesian CART. The predictive performance of TARGET is slightly worse than that of ensemble methods, such as random forests, but the TARGET solutions are far more interpretable.