Unsupervised and semi-supervised extreme learning machine with wavelet kernel for high dimensional data

Extreme learning machine (ELM) not only is an effective classifier in supervised learning, but also can be applied on unsupervised learning and semi-supervised learning. The model structure of unsupervised extreme learning machine (US-ELM) and semi-supervised extreme learning machine (SS-ELM) are same as ELM, the difference between them is the cost function. We introduce kernel function to US-ELM and propose unsupervised extreme learning machine with kernel (US-KELM). And SS-KELM has been proposed. Wavelet analysis has the characteristics of multivariate interpolation and sparse change, and Wavelet kernel functions have been widely used in support vector machine. Therefore, to realize a combination of the wavelet kernel function, US-ELM, and SS-ELM, unsupervised extreme learning machine with wavelet kernel function (US-WKELM) and semi-supervised extreme learning machine with wavelet kernel function (SS-WKELM) are proposed in this paper. The experimental results show the feasibility and validity of US-WKELM and SS-WKELM in clustering and classification.     

A novel single-loop procedure for time-variant reliability analysis based on Kriging model

This paper proposes a novel single-loop procedure for time-variant reliability analysis based on a Kriging model. A new strategy is presented to decouple the double-loop Kriging model for time-variant reliability analysis, in which the extreme value response in double-loop procedure is replaced by the best value in the current sampled points to avoid the inner optimization loop. Consequently, the extreme value response surface for time-variant reliability analysis can be directly established through a single-loop Kriging surrogate model. To further improve the accuracy of the proposed Kriging model, two methods are provided to adaptively choose a new sample point for updating the model. One method is to apply two commonly used learning functions to select the new sample point that resides as close to the extreme value response surface as possible, and the other is to apply a new learning function to select the new point. Synchronously, the corresponding different stopping criteria are also provided. It is worth nothing that the proposed single-loop Kriging model for time-variant reliability analysis is for a single time-variant performance function. To verify the proposed method, it is applied to four examples, two of which have with random process and others have not. Other popular methods for time-variant reliability analysis including the existing single-loop Kriging model are also used for the comparative analysis and their results testify the effectiveness of the proposed method.     

A kernel extreme learning machine algorithm based on improved particle swam optimization

Kernel extreme learning machine (KELM) increases the robustness of extreme learning machine (ELM) by turning linearly non-separable data in a low dimensional space into a linearly separable one. However, the internal power parameters of ELM are initialized at random, causing the algorithm to be unstable. In this paper, we use the active operators particle swam optimization algorithm (APSO) to obtain an optimal set of initial parameters for KELM, thus creating an optimal KELM classifier named as APSO-KELM. Experiments on standard genetic datasets show that APSO-KELM has higher classification accuracy when being compared to the existing ELM, KELM, and these algorithms combining PSO/APSO with ELM/KELM, such as PSO-KELM, APSO-ELM, PSO-ELM, etc. Moreover, APSO-KELM has good stability and convergence, and is shown to be a reliable and effective classification algorithm.     

On the rate of convergence for multi-category classification based on convex losses

The multi-category classification algorithms play an important role in both theory and practice of machine learning.In this paper,we consider an approach to the multi-category classification based on minimizing a convex surrogate of the nonstandard misclassification loss.We bound the excess misclassification error by the excess convex risk.We construct an adaptive procedure to search the classifier and furthermore obtain its convergence rate to the Bayes rule.     

基于非对称半监督集成SVM的托攻击检测方法

受推荐系统在电子商务领域重大经济利益的驱动,恶意用户以非法牟利为目的实施托攻击,操纵改变推荐结果,使推荐系统面临严峻的信息安全威胁,如何识别和检测托攻击成为保障推荐系统信息安全的关键。传统支持向量机(SVM)方法同时受到小样本和数据不均衡两个问题的制约。为此,提出一种半监督SVM和非对称集成策略相结合的托攻击检测方法。首先训练初始SVM,然后引入K最近邻法优化分类面附近样本的标记质量,利用标记数据和未标记数据的混合样本集减少对标记数据的需求。最后,设计一种非对称加权集成策略,重点关注攻击样本的分类准确率,降低集成分类器对数据不均衡的敏感性。实验结果表明,本文方法有效地解决了小样本问题和数据不均衡分布问题,获得了较好的检测效果。     

Evolving fuzzy rule based controllers using genetic algorithms

The synthesis of genetics-based machine learning and fuzzy logic is beginning to show promise as a potent tool in solving complex control problems in multi-variate non-linear systems. In this paper an overview of current research applying the genetic algorithm to fuzzy rule based control is presented. A novel approach to genetics-based machine learning of fuzzy controllers, called a Pittsburgh Fuzzy Classifier System # 1 (P-FCS1) is proposed. P-FCS1 is based on the Pittsburgh model of learning classifier systems and employs variable length rule-sets and simultaneously evolves fuzzy set membership functions and relations. A new crossover operator which respects the functional linkage between fuzzy rules with overlapping input fuzzy set membership functions is introduced. Experimental results using P-FCS 1 are reported and compared with other published results. Application of P-FCS1 to a distributed control problem (dynamic routing in computer networks) is also described and experimental results are presented.     

基于GA-ELM的水上交通事故严重程度影响因素识别研究

水上交通事故严重程度影响因素的识别对从根本上减少严重事故件数、降低事故危害和损失具有重要意义。在历史事故报告的基础上,构建并量化事故影响因素集,提出以极限学习机(ELM)为一般事故、严重事故的二分类器,以遗传算法(GA)为因素搜索算法的GA-ELM因素识别模型。对发生在我国水域的737件水上交通事故进行实证研究,并与以支持向量机(SVM)为分类器的GA-SVM模型进行对比分析。结果表明,GA-ELM模型识别出时段、人为致因、环境致因等9个事故严重程度影响因素,较GA-SVM模型结果更为精简,且分类精度较不做因素识别时分别提高8.2%、7.1%。此外,GA-ELM大大缩短运算时间。由此可见,GA-ELM可为水上交通事故严重程度影响因素识别提供一个较好的方法。     

基于提升小波的基因芯片数据的分类预测

针对肿瘤的早期诊断,提出了一种基于提升小波变换的特征提取的方法,对肿瘤数据样本进行分析鉴别.该方法利用提升小波变换对190例肝癌（包括对照）和107例肺癌（包括对照）基因表达谱芯片数据进行处理后,提取信号的低频信息,经支持向量机训练学习,构造分类器模型,用于癌和非癌样本的区分甄别.实验结果表明,经提升小波变换提取的特征基因,送入分类器中能得到较高的分类率,且在支持向量机中选取线性核函数或径向基函数都能达到较好的分类效果.通过随机选取的20例基因表达谱芯片样本,对所建立的模型进行了测试,获得了很好的效果,因此,本文提出的方法对肿瘤的诊断有一定的应用意义.     

Transient landing dynamics analysis for a lunar lander with random and interval fields

This paper presents an objective comparison of random fields and interval fields to propagate spatial uncertainty, based on a finite element model of a lunar lander. The impulse based substructuring method is used to improve the analysis efficiency. The spatially uncertain input parameters are modeled by both random fields and interval fields. The objective of this work is to compare the applicability of both approaches in an early design stage under scarce information regarding the occurring spatial parameter variability. Focus is on the definition of the input side of the problem under this scarce knowledge, as well as the interpretation of the analysis outcome. To obtain an objective comparison between both approaches, the gradients in the interval field are tuned towards the gradients present in the random field. The result shows a very similar dependence and correlation structure between the local properties for both approaches. Furthermore, through the transient dynamic estimation, it is shown that the response ranges that are predicted by the interval field and random field are very close to each other.     

在线虚假评论识别中的数据贫乏问题研究

机器学习相关技术的发展提升了在线虚假评论识别的准确率,然而现阶段机器学习模型缺少足够量的已标注数据来进行模型训练。本文基于生成式对抗网络(GAN)提出了评论数据集扩充方法GAN-RDE(GAN-Review Dataset Expansion)以解决虚假评论识别中模型训练数据贫乏问题。具体而言,首先将初始评论数据划分为真实评论数据集和虚假评论数据集,使用真实评论数据集和虚假评论数据集分别训练GAN,生成符合真实评论与虚假评论特征分布的向量。然后将GAN训练得到的符合评论特征分布的向量与初始评论数据集的特征词词向量矩阵进行合并,扩充模型训练数据。最后,利用朴素贝叶斯、多层感知机和支持向量机作为基础分类器,对比数据扩充前后虚假评论识别的效果。实验结果表明,使用GAN-RDE方法扩充评论数据集后,机器学习模型对虚假评论识别准确率得到显著提升。     

Willingness-to-pay estimation with choice-based conjoint analysis: Addressing extreme response behavior with individually adapted designs

The increasing consideration of behavioral aspects in operations management models has prompted greater use of choice-based conjoint (CBC) studies in operations research. Such studies can elicit consumers’ willingness to pay (WTP), a core input for many optimization models. However, optimization models can yield valid results only if consumers’ WTP is estimated accurately. A simulation study and two field studies show that extreme response behavior in CBC studies, such that consumers always or never choose the no-purchase option, harms the validity of WTP estimates. Reporting the share of consumers who always and never select the no-purchase option allows for detecting extreme response behavior. This study suggests an individually adapted design that avoids extreme response behavior and thus significantly improves WTP estimation accuracy.     

Fluid sensing using microcantilevers: From physics-based modeling to deep learning

In-situ measurements of the viscosity and density of small volumes of liquids are required in several industrial applications. MEMS sensors deploying vibrating microstructures constitute an attractive alternative given the significant impact of the surrounding liquid on their dynamic behavior. In this work, we combine physics-based modeling approaches and deep learning techniques to simultaneously estimate the density and viscosity of liquids from the resonance frequencies and quality factors of immersed microcantilevers. The physics-based model is first validated by comparing the simulated resonance frequencies and quality factors of immersed microcantilevers to those obtained from experiments conducted on a large variety of liquids. Then, we use the simulations results to train deep neutral networks to learn the mapping from the data space to the parameter space. The deep learning method shows high prediction accuracy provided that there is enough independent input data, shows no bias in the predicted values, and provides the results instantaneously. The optimal accuracy in the estimation of the liquid viscosity and density is achieved when the first resonance frequency and corresponding quality factor are used as inputs.     

Sparse optimization in feature selection: application in neuroimaging

Feature selection plays an important role in the successful application of machine learning techniques to large real-world datasets. Avoiding model overfitting, especially when the number of features far exceeds the number of observations, requires selecting informative features and/or eliminating irrelevant ones. Searching for an optimal subset of features can be computationally expensive. Functional magnetic resonance imaging (fMRI) produces datasets with such characteristics creating challenges for applying machine learning techniques to classify cognitive states based on fMRI data. In this study, we present an embedded feature selection framework that integrates sparse optimization for regularization (or sparse regularization) and classification. This optimization approach attempts to maximize training accuracy while simultaneously enforcing sparsity by penalizing the objective function for the coefficients of the features. This process allows many coefficients to become zero, which effectively eliminates their corresponding features from the classification model. To demonstrate the utility of the approach, we apply our framework to three different real-world fMRI datasets. The results show that regularized classifiers yield better classification accuracy, especially when the number of initial features is large. The results further show that sparse regularization is key to achieving scientifically-relevant generalizability and functional localization of classifier features. The approach is thus highly suited for analysis of fMRI data.     

基于指数Laplace损失函数的回归估计鲁棒超限学习机

实际问题的数据集通常受到各种噪声的影响,超限学习机(extreme learning machine, ELM)对这类数据集进行学习时,表现出预测精度低、预测结果波动大.为了克服该缺陷,采用了能够削弱噪声影响的指数Laplace损失函数.该损失函数是建立在Gauss核函数基础上,具有可微、非凸、有界且能够趋近于Laplace函数的特点.将其引入到超限学习机中,提出了鲁棒超限学习机回归估计(exponential Laplace loss function based robust ELM for regression, ELRELM)模型.利用迭代重赋权算法求解模型的优化问题.在每次迭代中,噪声样本点被赋予较小的权值,能够有效地提高预测精度.真实数据集实验验证了所提出的模型相比较于对比算法具有更优的学习性能和鲁棒性.     

Denoising deep extreme learning machine for sparse representation

In recent years, a great deal of research has focused on the sparse representation for signal. Particularly, a dictionary learning algorithm, K-SVD, is introduced to efficiently learn an redundant dictionary from a set of training signals. Indeed, much progress has been made in different aspects. In addition, there is an interesting technique named extreme learning machine (ELM), which is an single-layer feed-forward neural networks (SLFNs) with a fast learning speed, good generalization and universal classification capability. In this paper, we propose an optimization method about K-SVD, which is an denoising deep extreme learning machines based on autoencoder (DDELM-AE) for sparse representation. In other words, we gain a new learned representation through the DDELM-AE and as the new “input”, it makes the conventional K-SVD algorithm perform better. To verify the classification performance of the new method, we conduct extensive experiments on real-world data sets. The performance of the deep models (i.e., Stacked Autoencoder) is comparable. The experimental results indicate the fact that our proposed method is very efficient in the sight of speed and accuracy.     

分类问题的在线顺序双并联快速学习机

双并联前馈神经网络模型是单层感知机和单隐层前馈神经网络的混合结构,本文构造了一种双并联快速学习机算法,与其他类似算法比较,提出的算法能利用较少的隐层单元及更少的待定参数,获得近似的学习性能.数值实验表明,对很多实际分类问题,提出的算法具备更佳的泛化能力,因而可以作为快速学习机算法的有益补充.     

基于支持向量机的大豆产量预测研究

应用支持向量机(SVM)的算法进行中国大豆产量的预测研究,用1991-2008年中国大豆数据组成样本集,建立影响因素与大豆产量之间的SVM模型.利用SVM对输入和输出数据进行训练学习,逼近历史数据所隐含的函数关系,完成对新数据序列的映射关系,从而完成对未来年份大豆的预测,并与其它几种方法的预测效果进行比较.结果表明,SVM预测模型预测大豆产量的精度优于其它预测方法.     

Supervised learning algorithms for multi-class classification problems with partial class memberships

In several application domains such as biology, computer vision, social network analysis and information retrieval, multi-class classification problems arise in which data instances not simply belong to one particular class, but exhibit a partial membership to several classes. Existing machine learning or fuzzy set approaches for representing this type of fuzzy information mainly focus on unsupervised methods. In contrast, we present in this article supervised learning algorithms for classification problems with partial class memberships, where class memberships instead of crisp class labels serve as input for fitting a model to the data. Using kernel logistic regression (KLR) as a baseline method, first a basic one-versus-all approach is proposed, by replacing the binary-coded label vectors with [0,1]-valued class memberships in the likelihood. Subsequently, we use this KLR extension as base classifier to construct one-versus-one decompositions, in which partial class memberships are transformed and estimated in a pairwise manner. Empirical results on synthetic data and a real-world application in bioinformatics confirm that our approach delivers promising results. The one-versus-all method yields the best computational efficiency, while the one-versus-one methods are preferred in terms of predictive performance, especially when the observed class memberships are heavily unbalanced.     

Multi-fidelity design optimization of transonic airfoils using physics-based surrogate modeling and shape-preserving response prediction

A computationally efficient design methodology for transonic airfoil optimization has been developed. In the optimization process, a numerically cheap physics-based low-fidelity surrogate (the transonic small-disturbance equation) is used in lieu of an accurate, but computationally expensive, high-fidelity (the compressible Euler equations) simulation model. Correction of the low-fidelity model is achieved by aligning its corresponding airfoil surface pressure distribution with that of the high-fidelity model using a shape-preserving response prediction technique. The resulting method requires only a single high-fidelity simulation per iteration of the design process. The method is applied to airfoil lift maximization in two-dimensional inviscid transonic flow, subject to constraints on shock-induced pressure drag and airfoil cross-sectional area. The results showed that more than a 90% reduction in high-fidelity function calls was achieved when compared to direct high-fidelity model optimization using a pattern-search algorithm.     

Detecting a conditional extreme value model

In classical extreme value theory probabilities of extreme events are estimated assuming all the components of a random vector to be in a domain of attraction of an extreme value distribution. In contrast, the conditional extreme value model assumes a domain of attraction condition on a sub-collection of the components of a multivariate random vector. This model has been studied in Heffernan and Tawn (JRSS B 66(3):497–546, 2004), Heffernan and Resnick (Ann Appl Probab 17(2):537–571, 2007), and Das and Resnick (2009). In this paper we propose three statistics which act as tools to detect this model in a bivariate set-up. In addition, the proposed statistics also help to distinguish between two forms of the limit measure that is obtained in the model.