一种通用的基于梯度的SVM核参数选取算法

核函数的选取是SVM分类器选取的核心问题.核函数的自动选取既可以提高分类器的性能,又可以减少人为的干预.因此如何自动选取核函数已经成为SVM的热点问题,但是这个问题并没有获得很好的解决.近年来对核函数参数的自动选取的研究,特别是对基于梯度的优化算法的研究取得了一定的进展.提出了一种基于梯度的核函数选取的通用算法,并进行了实验.     

基于KMOD核函数的SVM方法在信用评分中的应用

本文介绍了支持向量分类机,并引入具有更好识别能力的KMOD核函数建立了SVM信用卡分类模型.利用澳大利亚和德国的信用卡数据进行了数值实验,结果表明该模型在分类准确率、支持向量方面优于基于RBF的SVM模型.     

Fuzzy clustering using multiple Gaussian kernels with optimized-parameters

In this paper, we propose a new kernel-based fuzzy clustering algorithm which tries to find the best clustering results using optimal parameters of each kernel in each cluster. It is known that data with nonlinear relationships can be separated using one of the kernel-based fuzzy clustering methods. Two common fuzzy clustering approaches are: clustering with a single kernel and clustering with multiple kernels. While clustering with a single kernel doesn’t work well with “multiple-density” clusters, multiple kernel-based fuzzy clustering tries to find an optimal linear weighted combination of kernels with initial fixed (not necessarily the best) parameters. Our algorithm is an extension of the single kernel-based fuzzy c-means and the multiple kernel-based fuzzy clustering algorithms. In this algorithm, there is no need to give “good” parameters of each kernel and no need to give an initial “good” number of kernels. Every cluster will be characterized by a Gaussian kernel with optimal parameters. In order to show its effective clustering performance, we have compared it to other similar clustering algorithms using different databases and different clustering validity measures.     

A group of knowledge-incorporated multiple criteria linear programming classifiers

Classification is a main data mining task, which aims at predicting the class label of new input data on the basis of a set of pre-classified samples. Multiple criteria linear programming (MCLP) is used as a classification method in the data mining area, which can separate two or more classes by finding a discriminate hyperplane. Although MCLP shows good performance in dealing with linear separable data, it is no longer applicable when facing with nonlinear separable problems. A kernel-based multiple criteria linear programming (KMCLP) model is developed to solve nonlinear separable problems. In this method, a kernel function is introduced to project the data into a higher-dimensional space in which the data will have more chance to be linear separable. KMCLP performs well in some real applications. However, just as other prevalent data mining classifiers, MCLP and KMCLP learn only from training examples. In the traditional machine learning area, there are also classification tasks in which data sets are classified only by prior knowledge, i.e. expert systems. Some works combine the above two classification principles to overcome the faults of each approach. In this paper, we provide our recent works which combine the prior knowledge and the MCLP or KMCLP model to solve the problem when the input consists of not only training examples, but also prior knowledge. Specifically, how to deal with linear and nonlinear knowledge in MCLP and KMCLP models is the main concern of this paper. Numerical tests on the above models indicate that these models are effective in classifying data with prior knowledge.     

Kernel-based fuzzy clustering and fuzzy clustering: A comparative experimental study

In this study, we present a comprehensive comparative analysis of kernel-based fuzzy clustering and fuzzy clustering. Kernel based clustering has emerged as an interesting and quite visible alternative in fuzzy clustering, however, the effectiveness of this extension vis-à-vis some generic methods of fuzzy clustering has neither been discussed in a complete manner nor the performance of clustering quantified through a convincing comparative analysis. Our focal objective is to understand the performance gains and the importance of parameter selection for kernelized fuzzy clustering. Generic Fuzzy C-Means (FCM) and Gustafson–Kessel (GK) FCM are compared with two typical generalizations of kernel-based fuzzy clustering: one with prototypes located in the feature space (KFCM-F) and the other where the prototypes are distributed in the kernel space (KFCM-K). Both generalizations are studied when dealing with the Gaussian kernel while KFCM-K is also studied with the polynomial kernel. Two criteria are used in evaluating the performance of the clustering method and the resulting clusters, namely classification rate and reconstruction error. Through carefully selected experiments involving synthetic and Machine Learning repository (http://archive.ics.uci.edu/beta/) data sets, we demonstrate that the kernel-based FCM algorithms produce a marginal improvement over standard FCM and GK for most of the analyzed data sets. It has been observed that the kernel-based FCM algorithms are in a number of cases highly sensitive to the selection of specific values of the kernel parameters.     

Support vector machine classification with indefinite kernels

We propose a method for support vector machine classification using indefinite kernels. Instead of directly minimizing or stabilizing a nonconvex loss function, our algorithm simultaneously computes support vectors and a proxy kernel matrix used in forming the loss. This can be interpreted as a penalized kernel learning problem where indefinite kernel matrices are treated as noisy observations of a true Mercer kernel. Our formulation keeps the problem convex and relatively large problems can be solved efficiently using the projected gradient or analytic center cutting plane methods. We compare the performance of our technique with other methods on several standard data sets.     

基于间隔迁移的多视角支持向量机

针对同一对象从不同途径或不同层面获得的特征数据被称为多视角数据. 多视角学习是利用事物的多视角数据进行建模求解的一种新的机器学习方法. 大量研究表明, 多视角数据共同学习可以显著提高模型的学习效果, 因此许多相关模型及算法被提出. 多视角学习一般需遵循一 致性原则和互补性原则. 基于一致性原则,Farquhar 等人成功地将支持向量机(Support Vector Machine, SVM)和核典型相关分析(Kernel Canonical Correlation Analysis, KCCA)整合成一个单独的优化问题, 提出SVM-2K模型. 但是, SVM-2K模型并未充分利用多视角数据间的互补信息. 因此, 在SVM-2K模型的基础之上, 提出了基于间隔迁移的多视角支持向量机模型(Margin transfer-based multi-view support vector machine, M^2SVM), 该模型同时满足多视角学习的一致性和互补 性两原则. 进一步地, 从一致性的角度对其进行理论分析, 并 与SVM-2K比较, 揭示了 M^2SVM 比SVM-2K 更为灵活. 最后, 在大量的多视角数据集上验证了M^2SVM模型的有效性.     

一类基于L0/1软间隔损失函数的低秩支持张量机

支持向量机作为基于向量空间的一种传统的机器学习方法,不能直接处理张量类型的数据,否则不仅破坏数据的空间结构,还会造成维度灾难及小样本问题。作为支持向量机的一种高阶推广,用于处理张量数据分类的支持张量机已经引起众多学者的关注,并应用于遥感成像、视频分析、金融、故障诊断等多个领域。与支持向量机类似,已有的支持张量机模型中采用的损失函数多为L_0/1函数的代理函数。将直接使用L_0/1这一本原函数作为损失函数,并利用张量数据的低秩性,建立针对二分类问题的低秩支持张量机模型。针对这一非凸非连续的张量优化问题,设计交替方向乘子法进行求解,并通过对模拟数据和真实数据进行数值实验,验证模型与算法的有效性。     

Tuning membership functions of kernel fuzzy classifiers by maximizing margins

We propose two methods for tuning membership functions of a kernel fuzzy classifier based on the idea of SVM (support vector machine) training. We assume that in a kernel fuzzy classifier a fuzzy rule is defined for each class in the feature space. In the first method, we tune the slopes of the membership functions at the same time so that the margin between classes is maximized under the constraints that the degree of membership to which a data sample belongs is the maximum among all the classes. This method is similar to a linear all-at-once SVM. We call this AAO tuning. In the second method, we tune the membership function of a class one at a time. Namely, for a class the slope of the associated membership function is tuned so that the margin between the class and the remaining classes is maximized under the constraints that the degrees of membership for the data belonging to the class are large and those for the remaining data are small. This method is similar to a linear one-against-all SVM. This is called OAA tuning. According to the computer experiment for fuzzy classifiers based on kernel discriminant analysis and those with ellipsoidal regions, usually both methods improve classification performance by tuning membership functions and classification performance by AAO tuning is slightly better than that by OAA tuning.     

Advances in credit scoring: combining performance and interpretation in kernel discriminant analysis

Due to the recent financial turmoil, a discussion in the banking sector about how to accomplish long term success, and how to follow an exhaustive and powerful strategy in credit scoring is being raised up. Recently, the significant theoretical advances in machine learning algorithms have pushed the application of kernel-based classifiers, producing very effective results. Unfortunately, such tools have an inability to provide an explanation, or comprehensible justification, for the solutions they supply. In this paper, we propose a new strategy to model credit scoring data, which exploits, indirectly, the classification power of the kernel machines into an operative field. A reconstruction process of the kernel classifier is performed via linear regression, if all predictors are numerical, or via a general linear model, if some or all predictors are categorical. The loss of performance, due to such approximation, is balanced by better interpretability for the end user, which is able to order, understand and to rank the influence of each category of the variables set in the prediction. An Italian bank case study has been illustrated and discussed; empirical results reveal a promising performance of the introduced strategy.     

基于PCA和改进的PSO-SVM的机器人移动方向分类

随着人们创新水平的不断提高,为了更加准确的实现机器人的导航任务,提出了一种基于改进的粒子群优化支持向量机中的参数的方法.首先利用主成分分析法对数据进行降维,然后利用改进的粒子群优化算法,对SVM中的惩罚参数c和核函数的参数g进行优化,最后代入到SVM中,以此来达到运用SVM对机器人的导航任务进行分类识别.相对于其他算法,容易发现改进的粒子群优化算法优化后的支持向量机可以达到很好的效果.这种识别分类可以帮助人们很好的对机器人进行导航,对今后机器人的研究具有很大的应用价值.     

Kernel Logistic Regression and the Import Vector Machine

The support vector machine (SVM) is known for its good performance in two-class classification, but its extension to multiclass classification is still an ongoing research issue. In this article, we propose a new approach for classification, called the import vector machine (IVM), which is built on kernel logistic regression (KLR). We show that the IVM not only performs as well as the SVM in two-class classification, but also can naturally be generalized to the multiclass case. Furthermore, the IVM provides an estimate of the underlying probability. Similar to the support points of the SVM, the IVM model uses only a fraction of the training data to index kernel basis functions, typically a much smaller fraction than the SVM. This gives the IVM a potential computational advantage over the SVM.     

Power Series Kernels

We introduce a class of analytic positive definite multivariate kernels which includes infinite dot product kernels as sometimes used in machine learning, certain new nonlinearly factorizable kernels, and a kernel which is closely related to the Gaussian. Each such kernel reproduces in a certain “native” Hilbert space of multivariate analytic functions. If functions from this space are interpolated in scattered locations by translates of the kernel, we prove spectral convergence rates of the interpolants and all derivatives. By truncation of the power series of the kernel-based interpolants, we constructively generalize the classical Bernstein theorem concerning polynomial approximation of analytic functions to the multivariate case. An application to machine learning algorithms is presented.      

装备费用预测中的混合核函数支持向量机

在支持向量机预测建模中,核函数用来将低维特征空间中的非线性问题映射为高维特征空间中的线性问题.核函数的特征对于支持向量机的学习和预测都有很重要的影响.考虑到两种典型核函数—全局核(多项式核函数)和局部核(RBF核函数)在拟合与泛化方面的特性,采用了一种基于混合核函数的支持向量机方法用于预测建模.为了评价不同核函数的建模效果、得到更好的预测性能,采用遗传算法自适应进化支持向量机模型的各项参数,并将其应用于装备费用预测的实际问题中.实际计算表明采用混合核函数的支持向量机较单一核函数时有更好的预测性能,可以作为一种有效的预测建模方法在装备管理中推广应用.     

SVM法分类:它的内容和挑战

§ 1　 IntroductionIf you knock the word“SVM”in the SCI index tool on International network,youwould take on thousands of records immediately.This shows its great effects on ourworld.SVM,namely,support vector machines have been successfully applied to a numberof applications ranging from particle identification and text categorization to engine knockdetection,bioinformatics and database marketing[1— 6] .The approach is systematic andproperly motivated by statistical learning theory[7] .…     

投影寻踪方法与高光谱遥感图像数据特征提取的研究

高光谱技术的发展与应用对高维的高光谱遥感数据的处理方法提出了更高的要求，投影寻踪方法是一种处理高维数据的十分有效的降维方法。高光谱图像分析对投影寻踪方法仍是一个全新的领域。本文介绍了投影寻踪的一般概念。分析了基于信息散度指标投影的寻踪方法在高光谱图像处理中的应用，给出了它与主成份分析处理结果的对比。并提出PP与高光谱研究将来的发展方向。     

一种加权的支持向量机及其在储层识别中的应用

标准支持向量机(SVM)抗噪声能力不强,当训练样本中存在有噪声或者野点时,会影响最优分类面的产生,最终导致分类结果出现偏差。针对这一问题,提出了一种考虑最小包围球的加权支持向量机(WSVM),给每个样本点赋予不同的权值,以此来降低噪声或野点对分类结果的影响。对江汉油田某区块的oilsk81,oilsk83和oilsk85三口油井的测井数据进行交叉验证,其中核函数采用了线性、指数和RBF这3种不同的核函数。测试结果显示,无论是在SVM还是在WSVM中,核函数选择RBF识别率都是最高的,同时提出的WSVM不受核函数的影响,识别稳定性好,且在交叉验证中识别率都能够达到100%。     

Nonlinear optimization and support vector machines

Support Vector Machine (SVM) is one of the most important class of machine learning models and algorithms, and has been successfully applied in various fields. Nonlinear optimization plays a crucial role in SVM methodology, both in defining the machine learning models and in designing convergent and efficient algorithms for large-scale training problems. In this paper we present the convex programming problems underlying SVM focusing on supervised binary classification. We analyze the most important and used optimization methods for SVM training problems, and we discuss how the properties of these problems can be incorporated in designing useful algorithms.     

MEA-BP模型在遥感影像分类中的应用研究

遥感影像分类作为遥感技术的一个重要应用,对遥感技术的发展具有重要作用.针对遥感影像数据特点,在目前的非线性研究方法中主要用到的是BP神经网络模型.但是BP神经网络模型存在对初始权阈值敏感、易陷入局部极小值和收敛速度慢的问题.因此,为了提高模型遥感影像分类精度,提出采用MEA-BP模型进行遥感影像数据分类.首先采用思维进化算法代替BP神经网络算法进行初始寻优,再用改进BP算法对优化的网络模型权阈值进一步精确优化,随后建立基于思维进化算法的BP神经网络分类模型,并将其应用到遥感影像数据分类研究中.仿真结果表明,新模型有效提高了遥感影像分类准确性,为遥感影像分类提出了一种新的方法,具有广泛研究价值.     

基于子空间多核学习的企业财务困境预测方法

财务指标的异构性是影响企业财务困境预测精度的重要因素,现有多核学习方法能够用于解决异构数据学习问题。本文首先介绍了子空间多核学习财务困境预测理论框架,在此基础上根据子空间学习的最大化方差准则、类别可分性最大化准则、非线性子空间映射原理,提出了三种子空间多核学习方法,分别为最大化方差投影子空间多核学习、类别可分性最大化子空间多核学习、非线性子空间多核学习。利用采集的我国上市公司数据进行实验,对比所提出的方法同现有代表性财务困境预测方法,并对实验结果进行分析。实验结果表明,本文提出的子空间多核学习财务困境预测框架行之有效,该框架下所构造的子空间多核学习预测方法能够有效地提升财务困境预测精度。