Mixture-Based Probabilistic Graphical Models for the Label Ranking Problem

The goal of the Label Ranking (LR) problem is to learn preference models that predict the preferred ranking of class labels for a given unlabeled instance. Different well-known machine learning algorithms have been adapted to deal with the LR problem. In particular, fine-tuned instance-based algorithms (e.g., k-nearest neighbors) and model-based algorithms (e.g., decision trees) have performed remarkably well in tackling the LR problem. Probabilistic Graphical Models (PGMs, e.g., Bayesian networks) have not been considered to deal with this problem because of the difficulty of modeling permutations in that framework. In this paper, we propose a Hidden Naive Bayes classifier (HNB) to cope with the LR problem. By introducing a hidden variable, we can design a hybrid Bayesian network in which several types of distributions can be combined: multinomial for discrete variables, Gaussian for numerical variables, and Mallows for permutations. We consider two kinds of probabilistic models: one based on a Naive Bayes graphical structure (where only univariate probability distributions are estimated for each state of the hidden variable) and another where we allow interactions among the predictive attributes (using a multivariate Gaussian distribution for the parameter estimation). The experimental evaluation shows that our proposals are competitive with the start-of-the-art algorithms in both accuracy and in CPU time requirements.     

Uncertainty analysis in matched-field geoacoustic inversions

Quantifying uncertainty for parameter estimates obtained from matched-field geoacoustic inversions using a Bayesian approach requires estimation of the uncertainties in the data due to ambient noise as well as modeling errors. In this study, the variance parameter of the Gaussian error model, hereafter called error variance, is assumed to describe the data uncertainty. In practice, this parameter is not known a priori, and choosing a particular value is often difficult. Hence, to account for the uncertainty in error variance, several methods are introduced for implementing both the full and empirical Bayesian approaches. A full Bayesian approach that permits uncertainty of the error variance to propagate through the parameter estimation processes is a natural way of incorporating the uncertainty of error variance. Due to the large number of unknown parameters in the full Bayesian uncertainty analysis, an alternative, the empirical Bayesian approach, is developed, in which the posterior distributions of model parameters are conditioned on a point estimate of the error variance. Comparisons between the full and empirical Bayesian inferences of model parameters are presented using both synthetic and experimental data.     

Data-free inference of the joint distribution of uncertain model parameters

A critical problem in accurately estimating uncertainty in model predictions is the lack of details in the literature on the correlation (or full joint distribution) of uncertain model parameters. In this paper we describe a framework and a class of algorithms for analyzing such “missing data” problems in the setting of Bayesian statistics. The analysis focuses on the family of posterior distributions consistent with given statistics (e.g. nominal values, confidence intervals). The combining of consistent distributions is addressed via techniques from the opinion pooling literature. The developed approach allows subsequent propagation of uncertainty in model inputs consistent with reported statistics, in the absence of data.     

Bayesian force reconstruction with an uncertain model

This paper presents a Bayesian approach for force reconstruction which can deal with both measurement noise and model uncertainty. In particular, an uncertain model is considered for inversion in the form of a matrix of frequency response functions whose modal parameters originate from either measurements or a finite element model. The model uncertainty and the regularization parameter are jointly determined with the unknown force through Monte Carlo Markov chain methods. Bayesian credible intervals of the force are built from its posterior probability density function by taking into account the quantified model uncertainty and measurement noise. The proposed approach is illustrated and validated on numerical and experimental examples.     

Cheminformatics analysis and learning in a data pipelining environment

Summary Workflow technology is being increasingly applied in discovery information to organize and analyze data. SciTegic's Pipeline Pilot is a chemically intelligent implementation of a workflow technology known as data pipelining. It allows scientists to construct and execute workflows using components that encapsulate many cheminformatics based algorithms. In this paper we review SciTegic's methodology for molecular fingerprints, molecular similarity, molecular clustering, maximal common subgraph search and Bayesian learning. Case studies are described showing the application of these methods to the analysis of discovery data such as chemical series and high throughput screening results. The paper demonstrates that the methods are well suited to a wide variety of tasks such as building and applying predictive models of screening data, identifying molecules for lead optimization and the organization of molecules into families with structural commonality.     

The Large Hadron Collider: A Marvel of Technology,edited by Lyndon Evans

Machine learning algorithms learn a desired input-output relation from examples in order to interpret new inputs. This is important for tasks such as image and speech recognition or strategy optimisation, with growing applications in the IT industry. In the last couple of years, researchers investigated if quantum computing can help to improve classical machine learning algorithms. Ideas range from running computationally costly algorithms or their subroutines efficiently on a quantum computer to the translation of stochastic methods into the language of quantum theory. This contribution gives a systematic overview of the emerging field of quantum machine learning. It presents the approaches as well as technical details in an accessible way, and discusses the potential of a future theory of quantum learning.     

A Bayesian tutorial for data assimilation

Data assimilation is the process by which observational data are fused with scientific information. The Bayesian paradigm provides a coherent probabilistic approach for combining information, and thus is an appropriate framework for data assimilation. Viewing data assimilation as a problem in Bayesian statistics is not new. However, the field of Bayesian statistics is rapidly evolving and new approaches for model construction and sampling have been utilized recently in a wide variety of disciplines to combine information. This article includes a brief introduction to Bayesian methods. Paying particular attention to data assimilation, we review linkages to optimal interpolation, kriging, Kalman filtering, smoothing, and variational analysis. Discussion is provided concerning Monte Carlo methods for implementing Bayesian analysis, including importance sampling, particle filtering, ensemble Kalman filtering, and Markov chain Monte Carlo sampling. Finally, hierarchical Bayesian modeling is reviewed. We indicate how this approach can be used to incorporate significant physically based prior information into statistical models, thereby accounting for uncertainty. The approach is illustrated in a simplified advection–diffusion model.     

Sentiment Analysis of Persian Movie Reviews Using Deep Learning

Sentiment analysis aims to automatically classify the subject’s sentiment (e.g., positive, negative, or neutral) towards a particular aspect such as a topic, product, movie, news, etc. Deep learning has recently emerged as a powerful machine learning technique to tackle the growing demand for accurate sentiment analysis. However, the majority of research efforts are devoted to English-language only, while information of great importance is also available in other languages. This paper presents a novel, context-aware, deep-learning-driven, Persian sentiment analysis approach. Specifically, the proposed deep-learning-driven automated feature-engineering approach classifies Persian movie reviews as having positive or negative sentiments. Two deep learning algorithms, convolutional neural networks (CNN) and long-short-term memory (LSTM), are applied and compared with our previously proposed manual-feature-engineering-driven, SVM-based approach. Simulation results demonstrate that LSTM obtained a better performance as compared to multilayer perceptron (MLP), autoencoder, support vector machine (SVM), logistic regression and CNN algorithms.     

基于重要度的分级贝叶斯网络诊断模型研究

为了提高复杂系统故障检测和诊断的准确性,从数据的不确定性、诊断的不确定性以及特征参数选择的不确定性三方面展开研究。在贝叶斯网络解决不确定性问题优势的基础上,提出了基于重要度的分级贝叶斯网络模型。以往基于贝叶斯网络的方法无法做到系统地选择参数,建立的模型都是全模型。虽然全模型没有遗漏关键的参数,但包含了很多不必要的参数,这些多余参数不仅会对诊断造成不良影响,导致拖尾效应。仿真结果表明,提出方法可以获得更佳的故障特征,有利于提高故障识别能力,验证了模型的有效性。     

基于机器学习的合金催化剂表面吸附能预测

近年来，机器学习方法逐渐成为多相催化中的一种关键研究手段. 二元合金材料作为重要的催化剂之一，在双功能催化剂的筛选中受到了广泛的关注. 本文提出了一个将机器学习方法应用在预测催化性质上的整体框架，从而快速预测原子、分子在金属和二元合金表面的吸附能. 通过测试不同的机器学习方法来评估它们对于该问题的适用性，并将树集成的方法与压缩感知方法相结合，利用约6×10⁴个吸附能数据构建了预测模型. 相对于线性比例关系，该方法可以更准确地预测大量合金上的吸附能(预测的均方根误差降低一半)，并且更通用地预测各种吸附物的能量，为发现新的双金属催化剂铺平了道路.     

Pulse coding off-chip learning algorithm for memristive artificial neural network

Memristive neural network has attracted tremendous attention since the memristor array can perform parallel multiply-accumulate calculation (MAC) operations and memory-computation operations as compared with digital CMOS hardware systems. However, owing to the variability of the memristor, the implementation of high-precision neural network in memristive computation units is still difficult. Existing learning algorithms for memristive artificial neural network (ANN) is unable to achieve the performance comparable to high-precision by using CMOS-based system. Here, we propose an algorithm based on off-chip learning for memristive ANN in low precision. Training the ANN in the high-precision in digital CPUs and then quantifying the weight of the network to low precision, the quantified weights are mapped to the memristor arrays based on VTEAM model through using the pulse coding weight-mapping rule. In this work, we execute the inference of trained 5-layers convolution neural network on the memristor arrays and achieve an accuracy close to the inference in the case of high precision (64-bit). Compared with other algorithms-based off-chip learning, the algorithm proposed in the present study can easily implement the mapping process and less influence of the device variability. Our result provides an effective approach to implementing the ANN on the memristive hardware platform.     

基于字典学习和稀疏表示的单通道语音增强算法综述*

如何从带噪语音信号中恢复出干净的语音信号一直都是信号处理领域的热点问题。近年来研究者相继提出了一些基于字典学习和稀疏表示的单通道语音增强算法,这些算法利用语音信号在时频域上的稀疏特性,通过学习训练数据样本的结构特征和规律来构造相应的字典,再对带噪语音信号进行投影以估计出干净语音信号。针对训练样本与测试数据不匹配的情况,有监督类的非负矩阵分解方法与基于统计模型的传统语音增强方法相结合,在增强阶段对语音字典和噪声字典进行更新,从而估计出干净语音信号。本文首先介绍了单通道情况下语音增强的信号模型,然后对4种典型的增强方法进行了阐述,最后对未来可能的研究热点进行了展望。     

Bayesian localization in an uncertain ocean environment

In order to improve the ability to localize a source in an uncertain acoustic environment,a Bayesian approach,referred to here as Bayesian localization is used by including the environment in the parameter search space.Genetic algorithms are used for the parameter optimization.This method integrates the a posterior probability density(PPD) over environmental parameters to obtain a sequence of marginal probability distributions over source range and depth,from which the most-probable source location and localization uncertainties can be extracted.Considering that the seabed density and attenuation are less sensitive to the objective function of matched field processing,we utilize the empirical relationship to invert those parameters indirectly.The broadband signals recorded by a vertical line array in a Yellow Sea experiment in 2000 are processed and analyzed.It was found that,the Bayesian localization method that incorporates the environmental variability into the processor,made it robust to the uncertainty in the ocean environment.In addition,using the empirical relationship could enhance the localization accuracy.     

Improved Local Search with Momentum for Bayesian Networks Structure Learning

Bayesian Networks structure learning (BNSL) is a troublesome problem that aims to search for an optimal structure. An exact search tends to sacrifice a significant amount of time and memory to promote accuracy, while the local search can tackle complex networks with thousands of variables but commonly gets stuck in a local optimum. In this paper, two novel and practical operators and a derived operator are proposed to perturb structures and maintain the acyclicity. Then, we design a framework, incorporating an influential perturbation factor integrated by three proposed operators, to escape current local optimal and improve the dilemma that outcomes trap in local optimal. The experimental results illustrate that our algorithm can output competitive results compared with the state-of-the-art constraint-based method in most cases. Meanwhile, our algorithm reaches an equivalent or better solution found by the state-of-the-art exact search and hybrid methods.     

An Interpretation Architecture for Deep Learning Models with the Application of COVID-19 Diagnosis

The Coronavirus disease 2019 (COVID-19) has become one of the threats to the world. Computed tomography (CT) is an informative tool for the diagnosis of COVID-19 patients. Many deep learning approaches on CT images have been proposed and brought promising performance. However, due to the high complexity and non-transparency of deep models, the explanation of the diagnosis process is challenging, making it hard to evaluate whether such approaches are reliable. In this paper, we propose a visual interpretation architecture for the explanation of the deep learning models and apply the architecture in COVID-19 diagnosis. Our architecture designs a comprehensive interpretation about the deep model from different perspectives, including the training trends, diagnostic performance, learned features, feature extractors, the hidden layers, the support regions for diagnostic decision, and etc. With the interpretation architecture, researchers can make a comparison and explanation about the classification performance, gain insight into what the deep model learned from images, and obtain the supports for diagnostic decisions. Our deep model achieves the diagnostic result of 94.75%, 93.22%, 96.69%, 97.27%, and 91.88% in the criteria of accuracy, sensitivity, specificity, positive predictive value, and negative predictive value, which are 8.30%, 4.32%, 13.33%, 10.25%, and 6.19% higher than that of the compared traditional methods. The visualized features in 2-D and 3-D spaces provide the reasons for the superiority of our deep model. Our interpretation architecture would allow researchers to understand more about how and why deep models work, and can be used as interpretation solutions for any deep learning models based on convolutional neural network. It can also help deep learning methods to take a step forward in the clinical COVID-19 diagnosis field.     

Useful multiparticle entanglement and sub-shot-noise sensitivity in experimental phase estimation

We experimentally demonstrate a general criterion to identify entangled states useful for the estimation of an unknown phase shift with a sensitivity higher than the shot-noise limit. We show how to exploit this entanglement on the examples of a maximum likelihood as well as of a Bayesian phase estimation protocol. Using an entangled four-photon state we achieve a phase sensitivity clearly beyond the shot-noise limit. Our detailed comparison of methods and quantum states for entanglement enhanced metrology reveals the connection between multiparticle entanglement and sub-shot-noise uncertainty, both in a frequentist and in a Bayesian phase estimation setting.     

基于大脑情感学习模型和自适应遗传算法的混沌时间序列预测

针对传统神经网络预测精度不高、收敛速度慢的问题,提出一种基于大脑情感学习模型和自适应遗传算法的混沌时间序列预测方法.大脑情感学习模型模拟了哺乳动物大脑中杏仁体和眶额皮质之间的情感学习机制,具有计算复杂度低、运算速度快的特点,因此可以大大提高混沌预测的快速性.为了进一步提高大脑情感学习模型的预测精度,采用自适应遗传算法优化其参数,将待优化的权值与阈值分布在染色体基因序列上,用适应度函数选出最佳参数,从而增强了模型的逼近能力.基于Lorenz混沌时间序列和实际地磁Dst指数序列的预测结果表明,本文方法较其他传统方法在预测精度、运算速度和稳定性上均具有明显优势.     

A Bayesian approach to matched field processing in uncertain ocean environments

An approach of Bayesian Matched Field Processing （MFP） was discussed in the uncertain ocean environment. In this approach, uncertainty knowledge is modeled and spatial and temporal data received by the array are fully used. Therefore, a mechanism for MFP is found, which well combines model-based and data-driven methods of uncertain field processing. By theoretical derivation, simulation analysis and the validation of the experimental array data at sea, we find that （1） the basic components of Bayesian matched field processors are the cor- responding sets of Bartlett matched field processor, MVDR （minimum variance distortionless response） matched field processor, etc.; （2） Bayesian MVDR/Bartlett MFP are the weighted sum of the MVDR/Bartlett MFP, where the weighted coefficients are the values of the a posteriori probability; （3） with the uncertain ocean environment, Bayesian MFP can more correctly locate the source than MVDR MFP or Bartlett MFP; （4） Bayesian MFP can better suppress sidelobes of the ambiguity surfaces.     

Radar sensor network for target detection using Chernoff information and relative entropy

In this paper, we propose to apply information theory to Ultra wide band (UWB) radar sensor network (RSN) to detect target in foliage environment. Information theoretic algorithms such as Maximum entropy method (MEM) and mutual information are proven methods, that can be applied to data collected by various sensors. However, the complexity of the environment poses uncertainty in fusion center. Chernoff information provides the best error exponent of detection in Bayesian environment. In this paper, we consider the target detection as binary hypothesis testing and use Chernoff information as sensor selection criterion, which significantly reduces the processing load. Another strong information theoretic algorithm, method of types, is applicable to our MEM based target detection algorithm as entropy is dependent on the empirical distribution only. Method of types analyzes the probability of a sequence based on empirical distribution. Based on this, we can find the bound on probability of detection. We also propose to use Relative entropy based processing in the fusion center based on method of types and Chernoff Stein Lemma. We study the required quantization level and number of nodes in gaining the best error exponent. The performance of the algorithms were evaluated, based on real world data.     

基于优化核极限学习机的风电功率时间序列预测

针对时间序列预测,在单隐层前馈神经网络的基础上,基于进化计算的优化策略,提出了一种优化的核极限学习机(optimized kernel extreme learning machine,O-KELM)方法.与极限学习机(extreme learning machine,ELM)方法相比,核极限学习机(kernel extreme learning machine,KELM)方法无须设定网络隐含层节点的数目,以核函数表示未知的隐含层非线性特征映射,通过正则化最小二乘算法计算网络的输出权值,它能以极快的学习速度获得良好的推广性.在KELM的基础上,分别将遗传算法、模拟退火、微分演化三种进化算法用于模型的结构输入选择、正则化系数以及核参数的优化选取,以进一步提高网络的性能.将O-KELM方法应用于标准Mackey-Glass混沌时间序列预测及某地区的风电功率时间序列预测实例中,在同等条件下,还与优化的极限学习机(optimized extreme learning machine,O-ELM)方法进行比较.实验结果表明,所提出的O-KELM方法在预测精度上优于O-ELM方法,表明了其有效性.