基于凸绝对值不等式的半监督最小二乘支持向量机

针对半监督分类问题,提出了基于凸绝对值不等式的半监督最小二乘支持向量机.传统的半监督支持向量机鲁棒性不强、效率不高,针对这些不足,利用凸绝对值不等式将平面分为两个有重叠的半平面,通过极小化重叠部分以及采用最小二乘支持向量机的思想处理无标签点,提高分类准确率,结果具有一定的鲁棒性.在8个数据集上进行了数值实验,说明了所提出的半监督分类算法的有效性.     

鲁棒半监督ν-支持向量分类机

支持向量机在近十年成为机器学习的主要学习技术,而且已经成功应用到有监督学习问题中。Fung和Mangasarian利用支持向量机对于既有已标类别样本又有未知类别样本的训练集进行训练,方法主要是利用少量已标明类别的样本进行训练得到一个分类器的同时对于未标明类别的样本进行分类,使得间隔最大化。此优化问题中假定样本是精确的,而在现实生活中,样本通常带有统计误差。因此,考虑样本带有扰动信息的半监督两类分类问题,给出鲁棒半监督v-支持向量分类算法。该算法的参数v易于选择,而数值试验也表明该算法具有良好的稳定性和较好的分类结果。     

两阶段半监督加权朴素信念分类模型

针对目前半监督分类算法中未考虑缺失属性隐含信息和算法复杂度高的情况,改进了朴素信念分类,提出了两阶段半监督加权朴素信念分类模型。与直推支持向量机对比实验结果表明两阶段半监督加权朴素信念分类模型减少了分类时间,并且在其能够明确分类的样本上的正确率与直推支持向量机相当,是一种有效的不完整数据分类算法。     

基于支持向量机混合采样的不平衡数据分类方法

利用传统支持向量机(SVM)对不平衡数据进行分类时,由于真实的少数类支持向量样本过少且难以被识别,造成了分类时效果不是很理想.针对这一问题,提出了一种基于支持向量机混合采样的不平衡数据分类方法(BSMS).该方法首先对经过支持向量机分类的原始不平衡数据按照所处位置的不同划分为支持向量区(SV),多数类非支持向量区(MNSV)以及少数类非支持向量区(FNSV)三个区域,并对MNSV区和FNSV区的样本做去噪处理;然后对SV区分类错误和部分分类正确且靠近决策边界的少数类样本重复进行过采样处理,直到找到测试结果最优的训练数据集;最后有选择的随机删除MNSV区的部分样本.实验结果表明:方法优于其他采样方法.     

基于Fuzzy理论的数据挖掘算法研究(Ⅰ)

“数据挖掘”是数据处理的一个新领域.支持向量机是数据挖掘的一种新方法,该技术在很多领域得到了成功的应用.但是,支持向量机目前还存在许多局限,当支持向量机的训练集中含有模糊信息时,支持向量机将无能为力.为解决一般情况下支持向量机中含有模糊信息(模糊参数)问题,研究了模糊机会约束规划、模糊分类中的模糊特征及其表示方法,建立了模糊支持向量分类机理论,给出了模糊线性可分的模糊支持向量分类机算法.     

基于支持向量机的商业银行对中小信贷企业选择方法的研究

引进了一种新的支持向量机的机器学习算法,解决了传统的机器学习算法在商业银行信贷企业选择方法上存在的局限性.以中小信贷企业为研究对象,运用支持向量机算法来解决分类问题和回归问题.通过某商业银行在中小信贷企业选择中的实际应用,并同神经网络训练得出的结果进行对比,证明这种支持向量机的机器学习算法,不仅具有较高的训练效率,而且有更高的精确度.     

基于Fuzzy理论的数据挖掘算法研究Ⅱ

"数据挖掘"是数据处理的一个新领域.支持向量机是数据挖掘的一种新方法,该技术在很多领域得到了成功的应用.但是,支持向量机目前还存在许多局限,当支持向量机的训练集中含有模糊信息时,支持向量机将无能为力.为解决一般情况下支持向量机中含有模糊信息(模糊参数)问题,研究了模糊机会约束规划、模糊分类中的模糊特征及其表示方法,建立了模糊支持向量分类机理论,给出了模糊线性可分的模糊支持向量分类机算法.     

一种改进二叉树支持向量机在故障诊断中的应用

考虑到构建二叉树支持向量机时样本的分布情况对分类器推广能力具有较大影响,提出一种改进的二叉树支持向量机层次结构构建方法.以类间样本距离和带权值的类内样本距离与其标准差的比值作为类的分类度.将类间距离大且类内样本平均分布广的类最先分离.利用标准数据集,通过与不同多类分类算法比较,验证了改进的二叉树支持向量机的优越性.对双转子涡喷发动机气路部件进行应用改进的算法进行故障诊断,得到了较好的故障识别率.     

非平衡数据集的支持向量域分类预测模型研究

基于非平衡数据集的支持向量域分类模型,提出了一种银行客户个人信用预测方法.首先分析了信用预测的主要方法及其不足,然后研究了支持向量域分类模型及其参数的非负二次规划乘性更新算法,进而提出基于支持向量域分类模型的银行客户个人信用预测方法,最后使用人工数据和实际数据对提出方法与支持向量机预测方法进行对比实验.实验结果表明对于银行客户个人信用预测的非平衡数据分析问题,基于支持向量域模型的分类预测方法更有效.     

平衡化图半监督学习方法

许多机器学习的实际应用中都存在数据不平衡问题,即某类的样本数目要远小于其他类别.数据不平衡会使得分类问题中的分类面过于倾向于适应大类而忽略小类,导致测试样本被错误地判断为大类.针对该问题,文章提出了一种平衡化图半监督学习方法.该方法在能量函数中引入均衡化因子项,使得置信值不仅在图上尽量光滑且在不同类别之间也尽量均衡,有效减小了数据不均衡的不利影响,21个标准数据集上对比实验的统计分析结果表明新方法在数据不平衡时具有显著(显著性水平为0.05)优于支持向量机以及其他图半监督学习方法的分类效果.     

Approximation of kernel matrices by circulant matrices and its application in kernel selection methods

This paper focuses on developing fast numerical algorithms for selection of a kernel optimal for a given training data set. The optimal kernel is obtained by minimizing a cost functional over a prescribed set of kernels. The cost functional is defined in terms of a positive semi-definite matrix determined completely by a given kernel and the given sampled input data. Fast computational algorithms are developed by approximating the positive semi-definite matrix by a related circulant matrix so that the fast Fourier transform can apply to achieve a linear or quasi-linear computational complexity for finding the optimal kernel. We establish convergence of the approximation method. Numerical examples are presented to demonstrate the approximation accuracy and computational efficiency of the proposed methods.     

Effective learning hyper-heuristics for the course timetabling problem

Course timetabling is an important and recurring administrative activity in most educational institutions. This article combines a general modeling methodology with effective learning hyper-heuristics to solve this problem. The proposed hyper-heuristics are based on an iterated local search procedure that autonomously combines a set of move operators. Two types of learning for operator selection are contrasted: a static (offline) approach, with a clear distinction between training and execution phases; and a dynamic approach that learns on the fly. The resulting algorithms are tested over the set of real-world instances collected by the first and second International Timetabling competitions. The dynamic scheme statistically outperforms the static counterpart, and produces competitive results when compared to the state-of-the-art, even producing a new best-known solution. Importantly, our study illustrates that algorithms with increased autonomy and generality can outperform human designed problem-specific algorithms.     

Designing greedy algorithms for the flow-shop problem by means of Empirically Adjusted Greedy Heuristics (EAGH)

This paper introduces Empirically Adjusted Greedy Heuristics (EAGH), a procedure for designing greedy algorithms for a given combinatorial optimization problem and illustrates the way in which EAGH works with an application to minimize the makespan in the permutation flow-shop problem. The basic idea behind EAGH is that a greedy heuristic can be seen as a member of an infinite set of heuristics, this set being defined by a function that depends on several parameters. Each set of values of the parameters corresponds to a specific greedy heuristic. Then, the best element of the set, for a training set of instances of the problem, is found by applying a non-linear optimization algorithm to a function that measures the quality of the obtained solutions to the instances of the training set, and which depends on the parameters that characterize each specific algorithm. EAGH allows improving known heuristics or finding good new ones.     

Parallel Synchronous and Asynchronous Space-Decomposition Algorithms for Large-Scale Minimization Problems

Three parallel space-decomposition minimization (PSDM) algorithms, based on the parallel variable transformation (PVT) and the parallel gradient distribution (PGD) algorithms (O.L. Mangasarian, SIMA Journal on Control and Optimization, vol. 33, no. 6, pp. 1916–1925.), are presented for solving convex or nonconvex unconstrained minimization problems. The PSDM algorithms decompose the variable space into subspaces and distribute these decomposed subproblems among parallel processors. It is shown that if all decomposed subproblems are uncoupled of each other, they can be solved independently. Otherwise, the parallel algorithms presented in this paper can be used. Numerical experiments show that these parallel algorithms can save processor time, particularly for medium and large-scale problems. Up to six parallel processors are connected by Ethernet networks to solve four large-scale minimization problems. The results are compared with those obtained by using sequential algorithms run on a single processor. An application of the PSDM algorithms to the training of multilayer Adaptive Linear Neurons (Madaline) and a new parallel architecture for such parallel training are also presented.     

Path relinking and GRG for artificial neural networks

Artificial neural networks (ANN) have been widely used for both classification and prediction. This paper is focused on the prediction problem in which an unknown function is approximated. ANNs can be viewed as models of real systems, built by tuning parameters known as weights. In training the net, the problem is to find the weights that optimize its performance (i.e., to minimize the error over the training set). Although the most popular method for training these networks is back propagation, other optimization methods such as tabu search or scatter search have been successfully applied to solve this problem. In this paper we propose a path relinking implementation to solve the neural network training problem. Our method uses GRG, a gradient-based local NLP solver, as an improvement phase, while previous approaches used simpler local optimizers. The experimentation shows that the proposed procedure can compete with the best-known algorithms in terms of solution quality, consuming a reasonable computational effort.     

Projected viscosity subgradient methods for variational inequalities with equilibrium problem constraints in Hilbert spaces

In this paper, we introduce and study some low computational cost numerical methods for finding a solution of a variational inequality problem over the solution set of an equilibrium problem in a real Hilbert space. The strong convergence of the iterative sequences generated by the proposed algorithms is obtained by combining viscosity-type approximations with projected subgradient techniques. First a general scheme is proposed, and afterwards two practical realizations of it are studied depending on the characteristics of the feasible set. When this set is described by convex inequalities, the projections onto the feasible set are replaced by projections onto half-spaces with the consequence that most iterates are outside the feasible domain. On the other hand, when the projections onto the feasible set can be easily computed, the method generates feasible points and can be considered as a generalization of Maingé’s method to equilibrium problem constraints. In both cases, the strong convergence of the sequences generated by the proposed algorithms is proven.     

双水平集逼近油藏模型特征的数值模拟方法

本文使用双水平集函数逼近油藏模型特征, 构造出Uzawas 算法进行数值模拟. 对于两相流渗透率的数值求解问题, 可以通过测量油井数据和地震波数据来实现. 将构造出来的带限制的最优化问题使用变异的Lagrange 方法求解. 如果使用双水平集函数逼近渗透率函数, 则需要对Lagrange 函数进行修正, 从而将带限制的最优化问题转化成无限制的最优化问题. 由于双水平集函数的优越性, 进一步构造出最速梯度下降Uzawas 算法和算子分裂格式Uzawas 算法进行求解对应的最优化子问题. 数值算例表明设计的算法是高效的、稳定的.     

A novel hybrid algorithm combining hunting search with harmony search algorithm for training neural networks

Neural networks (NNs) are one of the most widely used techniques for pattern classification. Owing to the most common back-propagation training algorithm of NN being extremely computationally intensive and it having some drawbacks, such as converging into local minima, many meta-heuristic algorithms have been applied to training of NNs. This paper presents a novel hybrid algorithm which is the integration of Harmony Search (HS) and Hunting Search (HuS) algorithms, called h_HS-HuS, in order to train Feed-Forward Neural Networks (FFNNs) for pattern classification. HS and HuS algorithms are recently proposed meta-heuristic algorithms inspired from the improvisation process of musicians and hunting of animals, respectively. Harmony search builds up the main structure of the hybrid algorithm, and HuS forms the pitch adjustment phase of the HS algorithm. The performance proposed algorithm is compared to conventional and meta-heuristic algorithms. Empirical tests are carried out by training NNs on nine widely used classification benchmark problems. The experimental results show that the proposed hybrid harmony-hunting algorithm is highly capable of training NNs.     

Selecting appropriate machine learning methods for digital soil mapping

Digital soil mapping (DSM) increasingly makes use of machine learning algorithms to identify relationships between soil properties and multiple covariates that can be detected across landscapes. Selecting the appropriate algorithm for model building is critical for optimizing results in the context of the available data. Over the past decade, many studies have tested different machine learning (ML) approaches on a variety of soil data sets. Here, we review the application of some of the most popular ML algorithms for digital soil mapping. Specifically, we compare the strengths and weaknesses of multiple linear regression (MLR), k-nearest neighbors (KNN), support vector regression (SVR), Cubist, random forest (RF), and artificial neural networks (ANN) for DSM. These algorithms were compared on the basis of five factors: (1) quantity of hyperparameters, (2) sample size, (3) covariate selection, (4) learning time, and (5) interpretability of the resulting model. If training time is a limitation, then algorithms that have fewer model parameters and hyperparameters should be considered, e.g., MLR, KNN, SVR, and Cubist. If the data set is large (thousands of samples) and computation time is not an issue, ANN would likely produce the best results. If the data set is small (<100), then Cubist, KNN, RF, and SVR are likely to perform better than ANN and MLR. The uncertainty in predictions produced by Cubist, KNN, RF, and SVR may not decrease with large datasets. When interpretability of the resulting model is important to the user, Cubist, MLR, and RF are more appropriate algorithms as they do not function as “black boxes.” There is no one correct approach to produce models for predicting the spatial distribution of soil properties. Nonetheless, some algorithms are more appropriate than others considering the nature of the data and purpose of mapping activity.     

Design and analysis of predictive sorting algorithms

The focus of this research is the class of sequential algorithms, called predictive sorting algorithms, for sorting a given set ofn elements using pairwise comparisons. The order in which these pairwise comparisons are made is defined by a fixed sequence of all unordered pairs of distinct integers {1, 2, ...,n} called a sort sequence. A predictive sorting algorithm associated with a sort sequence specifies pairwise comparisons of elements in the input set in the order defined by the sort sequence, except that the comparisons whose outcomes can be inferred from the preceding pairs of comparisons are not performed. In this paper predictive sorting algorithms are obtained, based on known sorting algorithms, and are shown to be required on the averageO(n logn) comparisons.