基于间隔迁移的多视角支持向量机

针对同一对象从不同途径或不同层面获得的特征数据被称为多视角数据. 多视角学习是利用事物的多视角数据进行建模求解的一种新的机器学习方法. 大量研究表明, 多视角数据共同学习可以显著提高模型的学习效果, 因此许多相关模型及算法被提出. 多视角学习一般需遵循一 致性原则和互补性原则. 基于一致性原则,Farquhar 等人成功地将支持向量机(Support Vector Machine, SVM)和核典型相关分析(Kernel Canonical Correlation Analysis, KCCA)整合成一个单独的优化问题, 提出SVM-2K模型. 但是, SVM-2K模型并未充分利用多视角数据间的互补信息. 因此, 在SVM-2K模型的基础之上, 提出了基于间隔迁移的多视角支持向量机模型(Margin transfer-based multi-view support vector machine, M^2SVM), 该模型同时满足多视角学习的一致性和互补 性两原则. 进一步地, 从一致性的角度对其进行理论分析, 并 与SVM-2K比较, 揭示了 M^2SVM 比SVM-2K 更为灵活. 最后, 在大量的多视角数据集上验证了M^2SVM模型的有效性.     

Opening the ‘black box’ of simulations: increased transparency and effective communication through the systematic design of experiments

Many still view simulation models as a black box. This paper argues that perceptions could change if the systematic design of experiments (DOE) for simulation research was fully realized. DOE can increase (1) the transparency of simulation model behavior and (2) the effectiveness of reporting simulation results. Based on DOE principles, we develop a systematic procedure to guide the analysis of simulation models as well as concrete templates for sharing the results. A simulation model investigating the performance of learning algorithms in an economic mechanism design context illustrates our suggestions. Overall, the proposed systematic procedure for applying DOE principles complements current initiatives for a more standardized simulation research process.     

分类问题的在线顺序双并联快速学习机

双并联前馈神经网络模型是单层感知机和单隐层前馈神经网络的混合结构,本文构造了一种双并联快速学习机算法,与其他类似算法比较,提出的算法能利用较少的隐层单元及更少的待定参数,获得近似的学习性能.数值实验表明,对很多实际分类问题,提出的算法具备更佳的泛化能力,因而可以作为快速学习机算法的有益补充.     

Fixed-rank matrix factorizations and Riemannian low-rank optimization

Motivated by the problem of learning a linear regression model whose parameter is a large fixed-rank non-symmetric matrix, we consider the optimization of a smooth cost function defined on the set of fixed-rank matrices. We adopt the geometric framework of optimization on Riemannian quotient manifolds. We study the underlying geometries of several well-known fixed-rank matrix factorizations and then exploit the Riemannian quotient geometry of the search space in the design of a class of gradient descent and trust-region algorithms. The proposed algorithms generalize our previous results on fixed-rank symmetric positive semidefinite matrices, apply to a broad range of applications, scale to high-dimensional problems, and confer a geometric basis to recent contributions on the learning of fixed-rank non-symmetric matrices. We make connections with existing algorithms in the context of low-rank matrix completion and discuss the usefulness of the proposed framework. Numerical experiments suggest that the proposed algorithms compete with state-of-the-art algorithms and that manifold optimization offers an effective and versatile framework for the design of machine learning algorithms that learn a fixed-rank matrix.     

A genetic algorithm for the unrelated parallel machine scheduling problem with sequence dependent setup times

In this work a genetic algorithm is presented for the unrelated parallel machine scheduling problem in which machine and job sequence dependent setup times are considered. The proposed genetic algorithm includes a fast local search and a local search enhanced crossover operator. Two versions of the algorithm are obtained after extensive calibrations using the Design of Experiments (DOE) approach. We review, evaluate and compare the proposed algorithm against the best methods known from the literature. We also develop a benchmark of small and large instances to carry out the computational experiments. After an exhaustive computational and statistical analysis we can conclude that the proposed method shows an excellent performance overcoming the rest of the evaluated methods in a comprehensive benchmark set of instances.     

A neural fuzzy control system with structure and parameter learning

A general connectionist model, called neural fuzzy control network (NFCN), is proposed for the realization of a fuzzy logic control system. The proposed NFCN is a feedforward multilayered network which integrates the basic elements and functions of a traditional fuzzy logic controller into a connectionist structure which has distributed learning abilities. The NFCN can be constructed from supervised training examples by machine learning techniques, and the connectionist structure can be trained to develop fuzzy logic rules and find membership functions. Associated with the NFCN is a two-phase hybrid learning algorithm which utilizes unsupervised learning schemes for structure learning and the backpropagation learning scheme for parameter learning. By combining both unsupervised and supervised learning schemes, the learning speed converges much faster than the original backpropagation algorithm. The two-phase hybrid learning algorithm requires exact supervised training data for learning. In some real-time applications, exact training data may be expensive or even impossible to obtain. To solve this problem, a reinforcement neural fuzzy control network (RNFCN) is further proposed. The RNFCN is constructed by integrating two NFCNs, one functioning as a fuzzy predictor and the other as a fuzzy controller. By combining a proposed on-line supervised structure-parameter learning technique, the temporal difference prediction method, and the stochastic exploratory algorithm, a reinforcement learning algorithm is proposed, which can construct a RNFCN automatically and dynamically through a reward-penalty signal (i.e., “good” or “bad” signal). Two examples are presented to illustrate the performance and applicability of the proposed models and learning algorithms.     

Quadratic Majorization for Nonconvex Loss with Applications to the Boosting Algorithm

Classical robust statistical methods dealing with noisy data are often based on modifications of convex loss functions. In recent years, nonconvex loss-based robust methods have been increasingly popular. A nonconvex loss can provide robust estimation for data contaminated with outliers. The significant challenge is that a nonconvex loss can be numerically difficult to optimize. This article proposes quadratic majorization algorithm for nonconvex (QManc) loss. The QManc can decompose a nonconvex loss into a sequence of simpler optimization problems. Subsequently, the QManc is applied to a powerful machine learning algorithm: quadratic majorization boosting algorithm (QMBA). We develop QMBA for robust classification (binary and multi-category) and regression. In high-dimensional cancer genetics data and simulations, the QMBA is comparable with convex loss-based boosting algorithms for clean data, and outperforms the latter for data contaminated with outliers. The QMBA is also superior to boosting when directly implemented to optimize nonconvex loss functions. Supplementary material for this article is available online.     

Scheduling jobs with truncated exponential learning functions

In this paper we consider the single machine scheduling problem with truncated exponential learning functions. By the truncated exponential learning functions, we mean that the actual job processing time is a function which depends not only on the total normal processing times of the jobs already processed but also on a control parameter. The use of the truncated function is to model the phenomenon that the learning of a human activity is limited. We show that even with the introduction of the proposed model to job processing times, several single machine problems remain polynomially solvable. For the following three objective functions, the total weighted completion time, the discounted total weighted completion time, the maximum lateness, we present heuristic algorithms according to the corresponding problems without truncated exponential learning functions. We also analyse the worst-case bound of our heuristic algorithms.     

Robust generalized eigenvalue classifier with ellipsoidal uncertainty

Uncertainty is a concept associated with data acquisition and analysis, usually appearing in the form of noise or measure error, often due to some technological constraint. In supervised learning, uncertainty affects classification accuracy and yields low quality solutions. For this reason, it is essential to develop machine learning algorithms able to handle efficiently data with imprecision. In this paper we study this problem from a robust optimization perspective. We consider a supervised learning algorithm based on generalized eigenvalues and we provide a robust counterpart formulation and solution in case of ellipsoidal uncertainty sets. We demonstrate the performance of the proposed robust scheme on artificial and benchmark datasets from University of California Irvine (UCI) machine learning repository and we compare results against a robust implementation of Support Vector Machines.     

基于故障树和混沌粒子群算法的锻压机床故障诊断方法

锻压机床由于生产效率高和材料利用率高的特点,被广泛应用于各领域.然而,锻压机床发生故障时,其故障种类繁多、故障数据量大,所以对锻压机床故障源的快速、准确诊断较困难.针对该问题,文章提出一种将故障树分析法和混沌粒子群算法相融合的方法,对锻压机床的故障源进行故障诊断.该方法是先通过故障树分析法对锻压机床的故障进行分析从而得到故障模式及其故障概率,然后由得到的故障模式和已知的故障维修经验分析归纳出故障模式的学习样本,再根据得到的故障概率运用混沌粒子群算法的遍历性快速、准确地诊断出锻压机床发生故障的精确位置.文章提出的方法以锻压机床的伺服系统为例进行了故障诊断实验,将该实验结果与遗传算法、粒子群算法进行对比.实验结果表明,文章的算法在锻压机床伺服系统的故障诊断中准确度更高、速度更快.     

Decision making with imprecise parameters

We analyze the impact of imprecise parameters on performance of an uncertainty-modeling tool presented in this paper. In particular, we present a reliable and efficient uncertainty-modeling tool, which enables dynamic capturing of interval-valued clusters representations sets and functions using well-known pattern recognition and machine learning algorithms. We mainly deal with imprecise learning parameters in identifying uncertainty intervals of membership value distributions and imprecise functions. In the experiments, we use the proposed system as a decision support tool for a production line process. Simulation results indicate that in comparison to benchmark methods such as well-known type-1 and type-2 system modeling tools, and statistical machine-learning algorithms, proposed interval-valued imprecise system modeling tool is more robust with less error.     

Parallelization Strategies for Rollout Algorithms

Rollout algorithms are innovative methods, recently proposed by Bertsekas et al. [3], for solving NP-hard combinatorial optimization problems. The main advantage of these approaches is related to their capability of magnifying the effectiveness of any given heuristic algorithm. However, one of the main limitations of rollout algorithms in solving large-scale problems is represented by their computational complexity. Innovative versions of rollout algorithms, aimed at reducing the computational complexity in sequential environments, have been proposed in our previous work [9]. In this paper, we show that a further reduction can be accomplished by using parallel technologies. Indeed, rollout algorithms have very appealing characteristics that make them suitable for efficient and effective implementations in parallel environments, thus extending their range of relevant practical applications.We propose two strategies for parallelizing rollout algorithms and we analyze their performance by considering a shared-memory paradigm. The computational experiments have been carried out on a SGI Origin 2000 with 8 processors, by considering two classical combinatorial optimization problems. The numerical results show that a good reduction of the execution time can be obtained by exploiting parallel computing systems.     

Searching for knowledge in a world flooded with facts

The wide availability of computer technology and large electronic storage media has led to an enormous proliferation of databases in almost every area of human endeavour. This naturally creates an intense demand for powerful methods and tools for data analysis. Current methods and tools are primarily oriented toward extracting numerical and statistical data characteristics. While such characteristics are very important and useful, they are often insufficient. A decision maker typically needs an interpretation of these findings, and this has to be done by a data analyst. With the growth in the amount and complexity of the data, making such interpretations is an increasingly difficult problem. As a potential solution, this paper advocates the development of methods for conceptual data analysis. Such methods aim at semi-automating the processes of determining high-level data interpretations, and discovering qualitative patterns in data. It is argued that these methods could be built on the basis of algorithms developed in the area of machine learning. An exemplary system utilizing such algorithms, INLEN, is discussed. The system integrates machine learning and statistical analysis techniques with database and expert system technologies. Selected capabilities of the system are illustrated by examples from implemented modules.     

Multiple kernel boosting framework based on information measure for classification

The performance of kernel-based method, such as support vector machine (SVM), is greatly affected by the choice of kernel function. Multiple kernel learning (MKL) is a promising family of machine learning algorithms and has attracted many attentions in recent years. MKL combines multiple sub-kernels to seek better results compared to single kernel learning. In order to improve the efficiency of SVM and MKL, in this paper, the Kullback–Leibler kernel function is derived to develop SVM. The proposed method employs an improved ensemble learning framework, named KLMKB, which applies Adaboost to learning multiple kernel-based classifier. In the experiment for hyperspectral remote sensing image classification, we employ feature selected through Optional Index Factor (OIF) to classify the satellite image. We extensively examine the performance of our approach in comparison to some relevant and state-of-the-art algorithms on a number of benchmark classification data sets and hyperspectral remote sensing image data set. Experimental results show that our method has a stable behavior and a noticeable accuracy for different data set.     

机器学习中的最优化模型与对偶

数学最优化是以数学的方式来刻画和找出问题最优解的一门学科.机器学习利用数据构造预测方法,并对这些方法进行研究.介绍了机器学习中与支持向量机和稀疏重构相关的最优化模型.在此基础上,给出了三个典型最优化模型的对偶问题,并详细地讨论了对偶在求解这些问题中的应用.     

A new memetic algorithm based on cellular learning automata for solving the vertex coloring problem

Vertex coloring problem is a combinatorial optimization problem in graph theory in which a color is assigned to each vertex of graph such that no two adjacent vertices have the same color. In this paper a new hybrid algorithm which is obtained from combination of cellular learning automata (CLA) and memetic algorithm (MA) is proposed for solving the vertex coloring problem. CLA is an effective probabilistic learning model combining cellular automata and learning automaton (LA). Irregular CLA (ICLA) is a generalization of CLA in which the restriction of rectangular grid structure in CLA is removed. The proposed algorithm is based on the irregular open CLA (IOCLA) that is an extension of ICLA in which the evolution of CLA is influenced by both local and global environments. Similar to other IOCLA-based algorithms, in the proposed algorithm, local environment is constituted by neighboring LAs of any cell and the global environment consists of a pool of memes in which each meme corresponds to a certain local search method. Each meme is represented by a set of LAs from which the history of the corresponding local search method can be extracted. To show the superiority of the proposed algorithm over some well-known algorithms, several computer experiments have been conducted. The results show that the proposed algorithm performs better than other methods in terms of running time of algorithm and the required number of colors.     

A comparative study of the leading machine learning techniques and two new optimization algorithms

We present here a computational study comparing the performance of leading machine learning techniques to that of recently developed graph-based combinatorial optimization algorithms (SNC and KSNC). The surprising result of this study is that SNC and KSNC consistently show the best or close to best performance in terms of their F₁-scores, accuracy, and recall. Furthermore, the performance of SNC and KSNC is considerably more robust than that of the other algorithms; the others may perform well on average but tend to vary greatly across data sets. This demonstrates that combinatorial optimization techniques can be competitive as compared to state-of-the-art machine learning techniques. The code developed for SNC and KSNC is publicly available.     

基于复杂网络上监督学习方法的研究综述

复杂网络已经成为复杂系统分析问题的通用方法,随着人工智能和机器学习的广泛兴起,越来越多的学者开始关注在复杂网络上进行机器学习。监督学习作为机器学习的一个重要组成部分,本文深入研究和总结了基于复杂网络的监督学习方法。首先,本文分别从复杂网络和监督学习的理论基础入手,明确了相似性函数和相异性函数的概念和测度方法,系统梳理了复杂网络的构建方法,并阐明了监督学习的概念及其在机器学习中的地位。其次,介绍了监督学习的几种常用算法,梳理了各种算法的研究现状。然后,提出了基于复杂网络监督学习方法未来关注方向。最后,说明了基于复杂网络监督学习方法的局限性,为相关学者的研究提供了参考。     

Combining metaheuristics with mathematical programming, constraint programming and machine learning

During the last years, interest on hybrid metaheuristics has risen considerably in the field of optimization and machine learning. The best results found for many optimization problems in science and industry are obtained by hybrid optimization algorithms. Combinations of optimization tools such as metaheuristics, mathematical programming, constraint programming and machine learning, have provided very efficient optimization algorithms. Four different types of combinations are considered in this paper: (i) Combining metaheuristics with complementary metaheuristics. (ii) Combining metaheuristics with exact methods from mathematical programming approaches which are mostly used in the operations research community. (iii) Combining metaheuristics with constraint programming approaches developed in the artificial intelligence community. (iv) Combining metaheuristics with machine learning and data mining techniques.     

Multi-armed bandit-based hyper-heuristics for combinatorial optimization problems

There are significant research opportunities in the integration of Machine Learning (ML) methods and Combinatorial Optimization Problems (COPs). In this work, we focus on metaheuristics to solve COPs that have an important learning component. These algorithms must explore a solution space and learn from the information they obtain in order to find high-quality solutions. Among the metaheuristics, we study Hyper-Heuristics (HHs), algorithms that, given a number of low-level heuristics, iteratively select and apply heuristics to a solution. The HH we consider has a Markov model to produce sequences of low-level heuristics, which we combine with a Multi-Armed Bandit Problem (MAB)-based method to learn its parameters. This work proposes several improvements to the HH metaheuristic that yields a better learning for solving problem instances. Specifically, this is the first work in HHs to present Exponential Weights for Exploration and Exploitation (EXP3) as a learning method, an algorithm that is able to deal with adversarial settings. We also present a case study for the Vehicle Routing Problem with Time Windows (VRPTW), for which we include a list of low-level heuristics that have been proposed in the literature. We show that our algorithms can handle a large and diverse list of heuristics, illustrating that they can be easily configured to solve COPs of different nature. The computational results indicate that our algorithms are competitive methods for the VRPTW (2.16% gap on average with respect to the best known solutions), demonstrating the potential of these algorithms to solve COPs. Finally, we show how algorithms can even detect low-level heuristics that do not contribute to finding better solutions to the problem.