Fractional Deng Entropy and Extropy and Some Applications

Deng entropy and extropy are two measures useful in the Dempster–Shafer evidence theory (DST) to study uncertainty, following the idea that extropy is the dual concept of entropy. In this paper, we present their fractional versions named fractional Deng entropy and extropy and compare them to other measures in the framework of DST. Here, we study the maximum for both of them and give several examples. Finally, we analyze a problem of classification in pattern recognition in order to highlight the importance of these new measures.     

Experimental Study and Comparison of Imbalance Ensemble Classifiers with Dynamic Selection Strategy

Imbalance ensemble classification is one of the most essential and practical strategies for improving decision performance in data analysis. There is a growing body of literature about ensemble techniques for imbalance learning in recent years, the various extensions of imbalanced classification methods were established from different points of view. The present study is initiated in an attempt to review the state-of-the-art ensemble classification algorithms for dealing with imbalanced datasets, offering a comprehensive analysis for incorporating the dynamic selection of base classifiers in classification. By conducting 14 existing ensemble algorithms incorporating a dynamic selection on 56 datasets, the experimental results reveal that the classical algorithm with a dynamic selection strategy deliver a practical way to improve the classification performance for both a binary class and multi-class imbalanced datasets. In addition, by combining patch learning with a dynamic selection ensemble classification, a patch-ensemble classification method is designed, which utilizes the misclassified samples to train patch classifiers for increasing the diversity of base classifiers. The experiments’ results indicate that the designed method has a certain potential for the performance of multi-class imbalanced classification.     

Approaching Adversarial Example Classification with Chaos Theory

Adversarial examples are one of the most intriguing topics in modern deep learning. Imperceptible perturbations to the input can fool robust models. In relation to this problem, attack and defense methods are being developed almost on a daily basis. In parallel, efforts are being made to simply pointing out when an input image is an adversarial example. This can help prevent potential issues, as the failure cases are easily recognizable by humans. The proposal in this work is to study how chaos theory methods can help distinguish adversarial examples from regular images. Our work is based on the assumption that deep networks behave as chaotic systems, and adversarial examples are the main manifestation of it (in the sense that a slight input variation produces a totally different output). In our experiments, we show that the Lyapunov exponents (an established measure of chaoticity), which have been recently proposed for classification of adversarial examples, are not robust to image processing transformations that alter image entropy. Furthermore, we show that entropy can complement Lyapunov exponents in such a way that the discriminating power is significantly enhanced. The proposed method achieves 65% to 100% accuracy detecting adversarials with a wide range of attacks (for example: CW, PGD, Spatial, HopSkip) for the MNIST dataset, with similar results when entropy-changing image processing methods (such as Equalization, Speckle and Gaussian noise) are applied. This is also corroborated with two other datasets, Fashion-MNIST and CIFAR 19. These results indicate that classifiers can enhance their robustness against the adversarial phenomenon, being applied in a wide variety of conditions that potentially matches real world cases and also other threatening scenarios.     

Learning to Classify DWDM Optical Channels from Tiny and Imbalanced Data

Applying machine learning algorithms for assessing the transmission quality in optical networks is associated with substantial challenges. Datasets that could provide training instances tend to be small and heavily imbalanced. This requires applying imbalanced compensation techniques when using binary classification algorithms, but it also makes one-class classification, learning only from instances of the majority class, a noteworthy alternative. This work examines the utility of both these approaches using a real dataset from a Dense Wavelength Division Multiplexing network operator, gathered through the network control plane. The dataset is indeed of a very small size and contains very few examples of “bad” paths that do not deliver the required level of transmission quality. Two binary classification algorithms, random forest and extreme gradient boosting, are used in combination with two imbalance handling methods, instance weighting and synthetic minority class instance generation. Their predictive performance is compared with that of four one-class classification algorithms: One-class SVM, one-class naive Bayes classifier, isolation forest, and maximum entropy modeling. The one-class approach turns out to be clearly superior, particularly with respect to the level of classification precision, making it possible to obtain more practically useful models.     

基于自适应无参经验小波变换和选择集成分类模型的运动想象

运动想象模式识别率的提高对脑机接口(BCI)技术的应用具有重要意义,本文采用自适应无参经验小波变换(APEWT)和选择集成分类模型相结合的方法提高脑电(EEG)信号的分类识别准确率.首先,通过APEWT将EEG信号分解成不同的模态;然后,使用最优模态重构后的信号计算其能量谱(ES)特征,使用最优模态分量计算其边际谱(MS)特征;最后,将不同时间段的ES特征和不同频段的MS特征输入到构建的选择集成分类模型中,从而得到其分类结果,并将该方法与其他4种组合方法进行比较.实验结果表明,本文方法具有较好分类准确率和实时性,其平均分类正确率高于其他4种方法,同时较近期使用相同数据的文献也有优势.本文为在线运动想象类BCI的应用提供了新的方法和思路.     

Deep Learning and Entropy-Based Texture Features for Color Image Classification

In the domain of computer vision, entropy—defined as a measure of irregularity—has been proposed as an effective method for analyzing the texture of images. Several studies have shown that, with specific parameter tuning, entropy-based approaches achieve high accuracy in terms of classification results for texture images, when associated with machine learning classifiers. However, few entropy measures have been extended to studying color images. Moreover, the literature is missing comparative analyses of entropy-based and modern deep learning-based classification methods for RGB color images. In order to address this matter, we first propose a new entropy-based measure for RGB images based on a multivariate approach. This multivariate approach is a bi-dimensional extension of the methods that have been successfully applied to multivariate signals (unidimensional data). Then, we compare the classification results of this new approach with those obtained from several deep learning methods. The entropy-based method for RGB image classification that we propose leads to promising results. In future studies, the measure could be extended to study other color spaces as well.     

Preference-Driven Classification Measure

Classification is one of the main problems of machine learning, and assessing the quality of classification is one of the most topical tasks, all the more difficult as it depends on many factors. Many different measures have been proposed to assess the quality of the classification, often depending on the application of a specific classifier. However, in most cases, these measures are focused on binary classification, and for the problem of many decision classes, they are significantly simplified. Due to the increasing scope of classification applications, there is a growing need to select a classifier appropriate to the situation, including more complex data sets with multiple decision classes. This paper aims to propose a new measure of classifier quality assessment (called the preference-driven measure, abbreviated p-d), regardless of the number of classes, with the possibility of establishing the relative importance of each class. Furthermore, we propose a solution in which the classifier’s assessment can be adapted to the analyzed problem using a vector of preferences. To visualize the operation of the proposed measure, we present it first on an example involving two decision classes and then test its operation on real, multi-class data sets. Additionally, in this case, we demonstrate how to adjust the assessment to the user’s preferences. The results obtained allow us to confirm that the use of a preference-driven measure indicates that other classifiers are better to use according to preferences, particularly as opposed to the classical measures of classification quality assessment.     

自适应增强方法在光谱自动分类中的应用

针对活动星系核(AGN)光谱中发射线的不同特征,在恢复到静止系状态后的光谱上截取具有有效特征的波段范围,采用自适应增强(Adaboost)的方法,对宽线和窄线AGNs进行特征融合的分类实验,经分析,确定了以Hα和[NⅡ]发射线为主的波段为宽线和窄线AGNs光谱的主要区别特征。再单独对Hα和[NⅡ]发射线为主的波段,用自适应增强的方法对其进行光谱分类。自适应增强方法在训练过程中不断地加入“弱分类器”,直到达到某个预定的足够小的误差率或一定的循环次数,最后构成的总体分类器的分类判决由这些“弱分类器”各自的判决结果的投票来决定。此方法不需要事先调节参数,且“弱分类器”的分类结果只需好于随机猜测,算法简单。实验证明,对于单独采用以Hα和[NⅡ]发射线为主的波段,自适应增强方法能达到较好的分类效果,从而可有效地应用于大型光谱巡天所产生的活动星系核光谱的自动分类中。     

水下高分辨率声图中小目标的深度网络分类方法

针对声成像数据缺少条件下的水下沉底小目标分类问题,提出一种深度网络分类算法。首先,采用高斯混合模型对声影区统计特性进行建模并提取声图阴影,在此基础上构建仿真数据集和真实数据集。将仿真数据集输入卷积神经网络进行训练,保留其特征提取部分,用于对真实数据集进行特征提取.重建网络分类部分并采用真实数据集的特征向量进行训练。结果表明,所提出的方法分类正确率可达88.24%,与6种对照方法相比平均分类正确率分别提升8.67%,20.47%,19.78%,11.59%,9.01%,11.58%。验证了所提出方法在小样本条件下具有较好对水下沉底小目标的分类能力。其学习曲线收敛到96.25%,仅比验证曲线高5.14%,说明在一定程度上缓解了过拟合问题。将改进的卷积神经网络应用于融合分类器,通过与逻辑回归分类器、支持向量机对目标进行分类并融合决策,正确率为93.33%,可进一步提高算法的正确率和稳定性.     

An Entropy-Based Architecture for Detection of Sepsis in Newborn Cry Diagnostic Systems

The acoustic characteristics of cries are an exhibition of an infant’s health condition and these characteristics have been acknowledged as indicators for various pathologies. This study focused on the detection of infants suffering from sepsis by developing a simplified design using acoustic features and conventional classifiers. The features for the proposed framework were Mel-frequency Cepstral Coefficients (MFCC), Spectral Entropy Cepstral Coefficients (SENCC) and Spectral Centroid Cepstral Coefficients (SCCC), which were classified through K-nearest Neighborhood (KNN) and Support Vector Machine (SVM) classification methods. The performance of the different combinations of the feature sets was also evaluated based on several measures such as accuracy, F1-score and Matthews Correlation Coefficient (MCC). Bayesian Hyperparameter Optimization (BHPO) was employed to tailor the classifiers uniquely to fit each experiment. The proposed methodology was tested on two datasets of expiratory cries (EXP) and voiced inspiratory cries (INSV). The highest accuracy and F-score were 89.99% and 89.70%, respectively. This framework also implemented a novel feature selection method based on Fuzzy Entropy (FE) as a final experiment. By employing FE, the number of features was reduced by more than 40%, whereas the evaluation measures were not hindered for the EXP dataset and were even enhanced for the INSV dataset. Therefore, it was deduced through these experiments that an entropy-based framework is successful for identifying sepsis in neonates and has the advantage of achieving high performance with conventional machine learning (ML) approaches, which makes it a reliable means for the early diagnosis of sepsis in deprived areas of the world.     

Highly Accurate Visual Method of Mars Terrain Classification for Rovers Based on Novel Image Features

It is important for Mars exploration rovers to achieve autonomous and safe mobility over rough terrain. Terrain classification can help rovers to select a safe terrain to traverse and avoid sinking and/or damaging the vehicle. Mars terrains are often classified using visual methods. However, the accuracy of terrain classification has been less than 90% in read operations. A high-accuracy vision-based method for Mars terrain classification is presented in this paper. By analyzing Mars terrain characteristics, novel image features, including multiscale gray gradient-grade features, multiscale edges strength-grade features, multiscale frequency-domain mean amplitude features, multiscale spectrum symmetry features, and multiscale spectrum amplitude-moment features, are proposed that are specifically targeted for terrain classification. Three classifiers, K-nearest neighbor (KNN), support vector machine (SVM), and random forests (RF), are adopted to classify the terrain using the proposed features. The Mars image dataset MSLNet that was collected by the Mars Science Laboratory (MSL, Curiosity) rover is used to conduct terrain classification experiments. The resolution of Mars images in the dataset is 256 × 256. Experimental results indicate that the RF classifies Mars terrain at the highest level of accuracy of 94.66%.     

Improved migration models of biogeography-based optimization for sonar dataset classification by using neural network

Classification of experimental datasets such as target and clutter in sonar applications is a complex and challenging problem. One of the most useful instrument to classify sonar datasets is Multi-Layer Perceptron Neural Network (MLP NN). In this paper, due to the optimally updating the weights and biases vector of the MLP NN, Biogeography-Based Optimization (BBO) is used to train the network. BBO has a fair ability to solve high-dimensional real-world problems (such as sonar dataset classification) by maintaining a suitable balance between exploration and exploitation phases. The performance of BBO is sensitive to the migration model, especially for high-dimensional problems. To improve the exploitation ability of BBO and to record the better results for classifying sonar dataset, we propose novel migration models such as exponential-logarithmic, and some improved migration models having different emigration and immigration mathematical functions. To validate the performance of the proposed classifiers, this network will classify three datasets with various sizes and complexities. The simulation results indicate that our newly proposed classifiers perform better than the other benchmark algorithms in addition to original BBO in terms of avoiding gets stuck in local minima, classification accuracy, and convergence speed.     

Performance of Bearing Ball Defect Classification Based on the Fusion of Selected Statistical Features

Among the existing bearing faults, ball ones are known to be the most difficult to detect and classify. In this work, we propose a diagnosis methodology for these incipient faults’ classification using time series of vibration signals and their decomposition. Firstly, the vibration signals were decomposed using empirical mode decomposition (EMD). Time series of intrinsic mode functions (IMFs) were then obtained. Through analysing the energy content and the components’ sensitivity to the operating point variation, only the most relevant IMFs were retained. Secondly, a statistical analysis based on statistical moments and the Kullback–Leibler divergence (KLD) was computed allowing the extraction of the most relevant and sensitive features for the fault information. Thirdly, these features were used as inputs for the statistical clustering techniques to perform the classification. In the framework of this paper, the efficiency of several family of techniques were investigated and compared including linear, kernel-based nonlinear, systematic deterministic tree-based, and probabilistic techniques. The methodology’s performance was evaluated through the training accuracy rate (TrA), testing accuracy rate (TsA), training time (Trt) and testing time (Tst). The diagnosis methodology has been applied to the Case Western Reserve University (CWRU) dataset. Using our proposed method, the initial EMD decomposition into eighteen IMFs was reduced to four and the most relevant features identified via the IMFs’ variance and the KLD were extracted. Classification results showed that the linear classifiers were inefficient, and that kernel or data-mining classifiers achieved 100% classification rates through the feature fusion. For comparison purposes, our proposed method demonstrated a certain superiority over the multiscale permutation entropy. Finally, the results also showed that the training and testing times for all the classifiers were lower than 2 s, and 0.2 s, respectively, and thus compatible with real-time applications.     

利用带无标签数据的双支持向量机对恒星光谱分类

恒星光谱分类是天文技术与方法领域一直关注的热点问题之一。随着观测设备持续运行和不断改进,人类获得的光谱数量与日俱增。这些海量光谱为人工处理带来了极大挑战。鉴于此,研究人员开始关注数据挖掘算法,并尝试对这些光谱进行数据挖掘。近年来,神经网络、自组织映射、关联规则等数据挖掘方法广泛应用于恒星光谱分类。在这些方法中,支持向量机(SVM)以其强大的学习能力和高效的分类性能而备受推崇。SVM的基本思想是试图在两类样本之间找到一个最优分类面将两类分开。SVM在求解时,通过将其最优化问题转化为具有(QP)形式的凸问题,进而得到全局最优解。尽管该方法在实际应用中表现优良,但为了进一步提高其分类能力,有的学者提出双支持向量机(TSVM)。该方法通过构造两个非平行的分类面将两类分开,每一类靠近某个分类面,而远离另一个分类面。TSVM的计算效率较之传统SVM提高近4倍,因此,自TSVM提出后便受到研究人员的持续关注,并出现若干改进算法。在恒星光谱分类中,一般分类算法都是根据历史观测光谱来建立分类模型,其中最关键的是对光谱进行人工标注,这项工作极为繁琐,且容易犯错。如何利用已标记的光谱以及部分无标签的光谱来建立分类模型显得尤为重要。因此,提出带无标签数据的双支持向量机(TSVMUD)用以实现对恒星光谱智能分类的目的。该方法首先将光谱分为训练数据集和测试数据集两部分;然后,在训练集上进行学习,得到分类依据;最后利用分类依据对测试集上的光谱进行验证。继承了双支持向量机的优势,更重要的是,在训练集上学习分类模型过程中,不仅考虑有标记的训练样本,也考虑部分未标记的样本。一方面提高了学习效率,另一方面得到更优的分类模型。在SDSS DR8恒星光谱数据集上的比较实验表明,与支持向量机SVM、双支持向量机TSVM以及K近邻(KNN)等传统分类方法相比,带无标签数据的双支持向量机TSVMUD具有更优的分类能力。然而,该方法亦存在一定的局限性,其中一大难题是其无法处理海量光谱数据。该工作将借鉴海量数据随机采样思想,利用大数据处理技术,来对所提方法在大数据环境下的适应性展开进一步研究。     

Bird Species Identification Using Spectrogram Based on Multi-Channel Fusion of DCNNs

Deep convolutional neural networks (DCNNs) have achieved breakthrough performance on bird species identification using a spectrogram of bird vocalization. Aiming at the imbalance of the bird vocalization dataset, a single feature identification model (SFIM) with residual blocks and modified, weighted, cross-entropy function was proposed. To further improve the identification accuracy, two multi-channel fusion methods were built with three SFIMs. One of these fused the outputs of the feature extraction parts of three SFIMs (feature fusion mode), the other fused the outputs of the classifiers of three SFIMs (result fusion mode). The SFIMs were trained with three different kinds of spectrograms, which were calculated through short-time Fourier transform, mel-frequency cepstrum transform and chirplet transform, respectively. To overcome the shortage of the huge number of trainable model parameters, transfer learning was used in the multi-channel models. Using our own vocalization dataset as a sample set, it is found that the result fusion mode model outperforms the other proposed models, the best mean average precision (MAP) reaches 0.914. Choosing three durations of spectrograms, 100 ms, 300 ms and 500 ms for comparison, the results reveal that the 300 ms duration is the best for our own dataset. The duration is suggested to be determined based on the duration distribution of bird syllables. As for the performance with the training dataset of BirdCLEF2019, the highest classification mean average precision (cmAP) reached 0.135, which means the proposed model has certain generalization ability.     

Synchronization and control in intrinsic and designed computation: an information-theoretic analysis of competing models of stochastic computation

We adapt tools from information theory to analyze how an observer comes to synchronize with the hidden states of a finitary, stationary stochastic process. We show that synchronization is determined by both the process's internal organization and by an observer's model of it. We analyze these components using the convergence of state-block and block-state entropies, comparing them to the previously known convergence properties of the Shannon block entropy. Along the way we introduce a hierarchy of information quantifiers as derivatives and integrals of these entropies, which parallels a similar hierarchy introduced for block entropy. We also draw out the duality between synchronization properties and a process's controllability. These tools lead to a new classification of a process's alternative representations in terms of minimality, synchronizability, and unifilarity.     

An Evaluation of Entropy Measures for Microphone Identification

Research findings have shown that microphones can be uniquely identified by audio recordings since physical features of the microphone components leave repeatable and distinguishable traces on the audio stream. This property can be exploited in security applications to perform the identification of a mobile phone through the built-in microphone. The problem is to determine an accurate but also efficient representation of the physical characteristics, which is not known a priori. Usually there is a trade-off between the identification accuracy and the time requested to perform the classification. Various approaches have been used in literature to deal with it, ranging from the application of handcrafted statistical features to the recent application of deep learning techniques. This paper evaluates the application of different entropy measures (Shannon Entropy, Permutation Entropy, Dispersion Entropy, Approximate Entropy, Sample Entropy, and Fuzzy Entropy) and their suitability for microphone classification. The analysis is validated against an experimental dataset of built-in microphones of 34 mobile phones, stimulated by three different audio signals. The findings show that selected entropy measures can provide a very high identification accuracy in comparison to other statistical features and that they can be robust against the presence of noise. This paper performs an extensive analysis based on filter features selection methods to identify the most discriminating entropy measures and the related hyper-parameters (e.g., embedding dimension). Results on the trade-off between accuracy and classification time are also presented.     

Loudness predicts prominence: fundamental frequency lends little

We explored a database covering seven dialects of British and Irish English and three different styles of speech to find acoustic correlates of prominence. We built classifiers, trained the classifiers on human prominence/nonprominence judgments, and then evaluated how well they behaved. The classifiers operate on 452 ms windows centered on syllables, using different acoustic measures. By comparing the performance of classifiers based on different measures, we can learn how prominence is expressed in speech. Contrary to textbooks and common assumption, fundamental frequency (f0) played a minor role in distinguishing prominent syllables from the rest of the utterance. Instead, speakers primarily marked prominence with patterns of loudness and duration. Two other acoustic measures that we examined also played a minor role, comparable to f0. All dialects and speaking styles studied here share a common definition of prominence. The result is robust to differences in labeling practice and the dialect of the labeler.     

Part-Aware Mask-Guided Attention for Thorax Disease Classification

Thorax disease classification is a challenging task due to complex pathologies and subtle texture changes, etc. It has been extensively studied for years largely because of its wide application in computer-aided diagnosis. Most existing methods directly learn global feature representations from whole Chest X-ray (CXR) images, without considering in depth the richer visual cues lying around informative local regions. Thus, these methods often produce sub-optimal thorax disease classification performance because they ignore the very informative pathological changes around organs. In this paper, we propose a novel Part-Aware Mask-Guided Attention Network (PMGAN) that learns complementary global and local feature representations from all-organ region and multiple single-organ regions simultaneously for thorax disease classification. Specifically, multiple innovative soft attention modules are designed to progressively guide feature learning toward the global informative regions of whole CXR image. A mask-guided attention module is designed to further search for informative regions and visual cues within the all-organ or single-organ images, where attention is elegantly regularized by automatically generated organ masks and without introducing computation during the inference stage. In addition, a multi-task learning strategy is designed, which effectively maximizes the learning of complementary local and global representations. The proposed PMGAN has been evaluated on the ChestX-ray14 dataset and the experimental results demonstrate its superior thorax disease classification performance against the state-of-the-art methods.