Parameter Analysis of Multiscale Two-Dimensional Fuzzy and Dispersion Entropy Measures Using Machine Learning Classification

Two-dimensional fuzzy entropy, dispersion entropy, and their multiscale extensions (MFuzzyEn2D and MDispEn2D, respectively) have shown promising results for image classifications. However, these results rely on the selection of key parameters that may largely influence the entropy values obtained. Yet, the optimal choice for these parameters has not been studied thoroughly. We propose a study on the impact of these parameters in image classification. For this purpose, the entropy-based algorithms are applied to a variety of images from different datasets, each containing multiple image classes. Several parameter combinations are used to obtain the entropy values. These entropy values are then applied to a range of machine learning classifiers and the algorithm parameters are analyzed based on the classification results. By using specific parameters, we show that both MFuzzyEn2D and MDispEn2D approach state-of-the-art in terms of image classification for multiple image types. They lead to an average maximum accuracy of more than 95% for all the datasets tested. Moreover, MFuzzyEn2D results in a better classification performance than that extracted by MDispEn2D as a majority. Furthermore, the choice of classifier does not have a significant impact on the classification of the extracted features by both entropy algorithms. The results open new perspectives for these entropy-based measures in textural analysis.     

Gait Stability Measurement by Using Average Entropy

Gait stability has been measured by using many entropy-based methods. However, the relation between the entropy values and gait stability is worth further investigation. A research reported that average entropy (AE), a measure of disorder, could measure the static standing postural stability better than multiscale entropy and entropy of entropy (EoE), two measures of complexity. This study tested the validity of AE in gait stability measurement from the viewpoint of the disorder. For comparison, another five disorders, the EoE, and two traditional metrics methods were, respectively, used to measure the degrees of disorder and complexity of 10 step interval (SPI) and 79 stride interval (SI) time series, individually. As a result, every one of the 10 participants exhibited a relatively high AE value of the SPI when walking with eyes closed and a relatively low AE value when walking with eyes open. Most of the AE values of the SI of the 53 diseased subjects were greater than those of the 26 healthy subjects. A maximal overall accuracy of AE in differentiating the healthy from the diseased was 91.1%. Similar features also exists on those 5 disorder measurements but do not exist on the EoE values. Nevertheless, the EoE versus AE plot of the SI also exhibits an inverted U relation, consistent with the hypothesis for physiologic signals.     

Investigation on Identifying Implicit Learning Event from EEG Signal Using Multiscale Entropy and Artificial Bee Colony

The way people learn will play an essential role in the sustainable development of the educational system for the future. Utilizing technology in the age of information and incorporating it into how people learn can produce better learners. Implicit learning is a type of learning of the underlying rules without consciously seeking or understanding the rules; it is commonly seen in small children while learning how to speak their native language without learning grammar. This research aims to introduce a processing system that can systematically identify the relationship between implicit learning events and their Encephalogram (EEG) signal characteristics. This study converted the EEG signal from participants while performing cognitive task experiments into Multiscale Entropy (MSE) data. Using MSE data from different frequency bands and channels as features, the system explored a wide range of classifiers and observed their performance to see how they classified the features related to participants’ performance. The Artificial Bee Colony (ABC) method was used for feature selection to improve the process to make the system more efficient. The results showed that the system could correctly identify the differences between participants’ performance using MSE data and the ABC method with 95% confidence.     

Variational Embedding Multiscale Sample Entropy: A Tool for Complexity Analysis of Multichannel Systems

Entropy-based methods have received considerable attention in the quantification of structural complexity of real-world systems. Among numerous empirical entropy algorithms, conditional entropy-based methods such as sample entropy, which are associated with amplitude distance calculation, are quite intuitive to interpret but require excessive data lengths for meaningful evaluation at large scales. To address this issue, we propose the variational embedding multiscale sample entropy (veMSE) method and conclusively demonstrate its ability to operate robustly, even with several times shorter data than the existing conditional entropy-based methods. The analysis reveals that veMSE also exhibits other desirable properties, such as the robustness to the variation in embedding dimension and noise resilience. For rigor, unlike the existing multivariate methods, the proposed veMSE assigns a different embedding dimension to every data channel, which makes its operation independent of channel permutation. The veMSE is tested on both stimulated and real world signals, and its performance is evaluated against the existing multivariate multiscale sample entropy methods. The proposed veMSE is also shown to exhibit computational advantages over the existing amplitude distance-based entropy methods.     

Intelligent Fault Diagnosis of Rolling-Element Bearings Using a Self-Adaptive Hierarchical Multiscale Fuzzy Entropy

The fuzzy-entropy-based complexity metric approach has achieved fruitful results in bearing fault diagnosis. However, traditional hierarchical fuzzy entropy (HFE) and multiscale fuzzy entropy (MFE) only excavate bearing fault information on different levels or scales, but do not consider bearing fault information on both multiple layers and multiple scales at the same time, thus easily resulting in incomplete fault information extraction and low-rise identification accuracy. Besides, the key parameters of most existing entropy-based complexity metric methods are selected based on specialist experience, which indicates that they lack self-adaptation. To address these problems, this paper proposes a new intelligent bearing fault diagnosis method based on self-adaptive hierarchical multiscale fuzzy entropy. On the one hand, by integrating the merits of HFE and MFE, a novel complexity metric method, named hierarchical multiscale fuzzy entropy (HMFE), is presented to extract a multidimensional feature matrix of the original bearing vibration signal, where the important parameters of HMFE are automatically determined by using the bird swarm algorithm (BSA). On the other hand, a nonlinear feature matrix classifier with strong robustness, known as support matrix machine (SMM), is introduced for learning the discriminant fault information directly from the extracted multidimensional feature matrix and automatically identifying different bearing health conditions. Two experimental results on bearing fault diagnosis show that the proposed method can obtain average identification accuracies of 99.92% and 99.83%, respectively, which are higher those of several representative entropies reported by this paper. Moreover, in the two experiments, the standard deviations of identification accuracy of the proposed method were, respectively, 0.1687 and 0.2705, which are also greater than those of the comparison methods mentioned in this paper. The effectiveness and superiority of the proposed method are verified by the experimental results.     

两相流流型多尺度熵及动力学特性分析

研究了几种典型非线性时间序列的多尺度熵特征,在此基础上分析了由插入式阵列电导传感器采集的144种流动条件下的垂直上升气液两相流电导波动信号.研究结果表明：利用小尺度下样本熵的变化速率特征可以分辨三种典型流型（泡状流、段塞流、混状流）,而大尺度下样本熵的波动特征可以反映各种流型的动力学特性.泡状流随机可变特性表现为大尺度下样本熵的高值及振荡特征;段塞流气塞与液塞的间歇性运动表现为大尺度下样本熵的低值及平稳性;混状流极不稳定的振荡运动特性表现为介于泡状流及段塞流之间的熵值特点,并在更大尺度时熵值逐渐接近泡状流 关键词： 样本熵 多尺度熵 气液两相流 动力学特性     

A New Method Based on Time-Varying Filtering Intrinsic Time-Scale Decomposition and General Refined Composite Multiscale Sample Entropy for Rolling-Bearing Feature Extraction

The early fault diagnosis of rolling bearings has always been a difficult problem due to the interference of strong noise. This paper proposes a new method of early fault diagnosis for rolling bearings with entropy participation. First, a new signal decomposition method is proposed in this paper: intrinsic time-scale decomposition based on time-varying filtering. It is introduced into the framework of complete ensemble intrinsic time-scale decomposition with adaptive noise (CEITDAN). Compared with traditional intrinsic time-scale decomposition, intrinsic time-scale decomposition based on time-varying filtering can improve frequency-separation performance. It has strong robustness in the presence of noise interference. However, decomposition parameters (the bandwidth threshold and B-spline order) have significant impacts on the decomposition results of this method, and they need to be artificially set. Aiming to address this problem, this paper proposes rolling-bearing fault diagnosis optimization based on an improved coyote optimization algorithm (COA). First, the minimal generalized refined composite multiscale sample entropy parameter was used as the objective function. Through the improved COA algorithm, optimal intrinsic time-scale decomposition parameters based on time-varying filtering that match the input signal are obtained. By analyzing generalized refined composite multiscale sample entropy (GRCMSE), whether the mode component is dominated by the fault signal is determined. The signal is reconstructed and decomposed again. Finally, the mode component with the highest energy in the central frequency band is selected for envelope spectrum variation for fault diagnosis. Lastly, simulated and experimental signals were used to verify the effectiveness of the proposed method.     

Multistage Segmentation of Prostate Cancer Tissues Using Sample Entropy Texture Analysis

In this study, a multistage segmentation technique is proposed that identifies cancerous cells in prostate tissue samples. The benign areas of the tissue are distinguished from the cancerous regions using the texture of glands. The texture is modeled based on wavelet packet features along with sample entropy values. In a multistage segmentation process, the mean-shift algorithm is applied on the pre-processed images to perform a coarse segmentation of the tissue. Wavelet packets are employed in the second stage to obtain fine details of the structured shape of glands. Finally, the texture of the gland is modeled by the sample entropy values, which identifies epithelial regions from stroma patches. Although there are three stages of the proposed algorithm, the computation is fast as wavelet packet features and sample entropy values perform robust modeling for the required regions of interest. A comparative analysis with other state-of-the-art texture segmentation techniques is presented and dice ratios are computed for the comparison. It has been observed that our algorithm not only outperforms other techniques, but, by introducing sample entropy features, identification of cancerous regions of tissues is achieved with 90% classification accuracy, which shows the robustness of the proposed algorithm.     

Reduced System Complexity of Heart Rate Dynamics in Patients with Hyperthyroidism: A Multiscale Entropy Analysis

Studying heart rate dynamics would help understand the effects caused by a hyperkinetic heart in patients with hyperthyroidism. By using a multiscale entropy (MSE) analysis of heart rate dynamics derived from one-channel electrocardiogram recording, we aimed to compare the system complexity of heart rate dynamics between hyperthyroid patients and control subjects. A decreased MSE complexity index (CI) computed from MSE analysis reflects reduced system complexity. Compared with the control subjects (n = 37), the hyperthyroid patients (n = 37) revealed a significant decrease (p < 0.001) in MSE CI (hyperthyroid patients 10.21 ± 0.37 versus control subjects 14.08 ± 0.21), sample entropy for each scale factor (from 1 to 9), and high frequency power (HF) as well as a significant increase (p < 0.001) in low frequency power (LF) in normalized units (LF%) and ratio of LF to HF (LF/HF). In conclusion, besides cardiac autonomic dysfunction, the system complexity of heart rate dynamics is reduced in hyperthyroidism. This finding implies that the adaptability of the heart rate regulating system is impaired in hyperthyroid patients. Additionally, it might explain the exercise intolerance experienced by hyperthyroid patients. In addition, hyperthyroid patients and control subjects could be distinguished by the MSE CI computed from MSE analysis of heart rate dynamics.     

Using Bidimensional Multiscale Entropy Analysis of Ultrasound Images to Assess the Effect of Various Walking Intensities on Plantar Soft Tissues

Walking performance is usually assessed by linear analysis of walking outcome measures. However, human movements consist of both linear and nonlinear complexity components. The purpose of this study was to use bidimensional multiscale entropy analysis of ultrasound images to evaluate the effects of various walking intensities on plantar soft tissues. Twelve participants were recruited to perform six walking protocols, consisting of three speeds (slow at 1.8 mph, moderate at 3.6 mph, and fast at 5.4 mph) for two durations (10 and 20 min). A B-mode ultrasound was used to assess plantar soft tissues before and after six walking protocols. Bidimensional multiscale entropy (MSE_2D) and the Complexity Index (CI) were used to quantify the changes in irregularity of the ultrasound images of the plantar soft tissues. The results showed that the CI of ultrasound images after 20 min walking increased when compared to before walking (CI₄: 0.39 vs. 0.35; CI₅: 0.48 vs. 0.43, p < 0.05). When comparing 20 and 10 min walking protocols at 3.6 mph, the CI was higher after 20 min walking than after 10 min walking (CI₄: 0.39 vs. 0.36, p < 0.05; and CI₅: 0.48 vs. 0.44, p < 0.05). This is the first study to use bidimensional multiscale entropy analysis of ultrasound images to assess plantar soft tissues after various walking intensities.     

Classification of Actigraphy Records from Bipolar Disorder Patients Using Slope Entropy: A Feasibility Study

Bipolar Disorder (BD) is an illness with high prevalence and a huge social and economic impact. It is recurrent, with a long-term evolution in most cases. Early treatment and continuous monitoring have proven to be very effective in mitigating the causes and consequences of BD. However, no tools are currently available for a massive and semi-automatic BD patient monitoring and control. Taking advantage of recent technological developments in the field of wearables, this paper studies the feasibility of a BD episodes classification analysis while using entropy measures, an approach successfully applied in a myriad of other physiological frameworks. This is a very difficult task, since actigraphy records are highly non-stationary and corrupted with artifacts (no activity). The method devised uses a preprocessing stage to extract epochs of activity, and then applies a quantification measure, Slope Entropy, recently proposed, which outperforms the most common entropy measures used in biomedical time series. The results confirm the feasibility of the approach proposed, since the three states that are involved in BD, depression, mania, and remission, can be significantly distinguished.     

Evaluating Temporal Correlations in Time Series Using Permutation Entropy,Ordinal Probabilities and Machine Learning

Time series analysis comprises a wide repertoire of methods for extracting information from data sets. Despite great advances in time series analysis, identifying and quantifying the strength of nonlinear temporal correlations remain a challenge. We have recently proposed a new method based on training a machine learning algorithm to predict the temporal correlation parameter, α, of flicker noise (FN) time series. The algorithm is trained using as input features the probabilities of ordinal patterns computed from FN time series, xαFN(t), generated with different values of α. Then, the ordinal probabilities computed from the time series of interest, x(t), are used as input features to the trained algorithm and that returns a value, αe, that contains meaningful information about the temporal correlations present in x(t). We have also shown that the difference, Ω, of the permutation entropy (PE) of the time series of interest, x(t), and the PE of a FN time series generated with α=αe, xαeFN(t), allows the identification of the underlying determinism in x(t). Here, we apply our methodology to different datasets and analyze how αe and Ω correlate with well-known quantifiers of chaos and complexity. We also discuss the limitations for identifying determinism in highly chaotic time series and in periodic time series contaminated by noise. The open source algorithm is available on Github.     

A Rolling Bearing Fault Classification Scheme Based on k-Optimized Adaptive Local Iterative Filtering and Improved Multiscale Permutation Entropy

The health condition of the rolling bearing seriously affects the operation of the whole mechanical system. When the rolling bearing parts fail, the time series collected in the field generally shows strong nonlinearity and non-stationarity. To obtain the faulty characteristics of mechanical equipment accurately, a rolling bearing fault detection technique based on k-optimized adaptive local iterative filtering (ALIF), improved multiscale permutation entropy (improved MPE), and BP neural network was proposed. In the ALIF algorithm, a k-optimized ALIF method based on permutation entropy (PE) is presented to select the number of ALIF decomposition layers adaptively. The completely average coarse-graining method was proposed to excavate more hidden information. The performance analysis of the simulation signal shows that the improved MPE can more accurately dig out the depth information of the time series, and the entropy value obtained is more consistent and stable. In the research application, rolling bearing time series are decomposed by k-optimized ALIF to obtain a certain number of intrinsic mode functions (IMFs). Then the improved MPE value of effective IMF is calculated and input into backpropagation (BP) neural network as the feature vector for automatic fault identification. The comparative analysis of simulation signals shows that this method can extract fault information effectively. At the same time, the experimental part shows that this scheme not only effectively extracts the fault features, but also realizes the classification and identification of different fault modes and faults of different degrees, which has a certain application prospect in the research and application direction of rolling bearing fault identification.     

Optimal Frequency and Amplitude of Vertical Viewpoint Oscillation for Improving Vection Strength and Reducing Neural Constrains on Gait

Inducing self-motion illusions referred as vection are critical for improving the sensation of walking in virtual environments (VE). Adding viewpoint oscillations to a constant forward velocity in VE is effective for improving vection strength under static conditions. However, the effects of oscillation frequency and amplitude on vection strength under treadmill walking conditions are still unclear. Besides, due to the visuomotor entrainment mechanism, these visual oscillations would affect gait patterns and be detrimental for achieving natural walking if not properly designed. This study was aimed at determining the optimal frequency and amplitude of vertical viewpoint oscillations for improving vection strength and reducing gait constraints. Seven subjects walked on a treadmill while watching a visual scene. The visual scene presented a constant forward velocity equal to the treadmill velocity with different vertical viewpoint oscillations added. Five oscillation patterns with different combinations of frequency and amplitude were tested. Subjects gave verbal ratings of vection strength. The mediolateral (M-L) center of pressure (CoP) complexity was calculated to indicate gait constraints. After the experiment, subjects were asked to give the best and the worst oscillation pattern based on their walking experience. The oscillation frequency and amplitude had strong positive correlations with vection strength. The M-L CoP complexity was reduced under oscillations with low frequency. The medium oscillation amplitude had greater M-L CoP complexity than the small and large amplitude. Besides, subjects preferred those oscillation patterns with large gait complexity. We suggested that the oscillation amplitude with largest M-L CoP complexity should first be chosen to reduce gait constraints. Then, increasing the oscillation frequency to improve vection strength until individual preference or the boundary of motion sickness. These findings provide important guidelines to promote the sensation of natural walking in VE.     

Fault Diagnosis of Power Transformer Based on Time-Shift Multiscale Bubble Entropy and Stochastic Configuration Network

In order to accurately diagnose the fault type of power transformer, this paper proposes a transformer fault diagnosis method based on the combination of time-shift multiscale bubble entropy (TSMBE) and stochastic configuration network (SCN). Firstly, bubble entropy is introduced to overcome the shortcomings of traditional entropy models that rely too heavily on hyperparameters. Secondly, on the basis of bubble entropy, a tool for measuring signal complexity, TSMBE, is proposed. Then, the TSMBE of the transformer vibration signal is extracted as a fault feature. Finally, the fault feature is inputted into the stochastic configuration network model to achieve an accurate identification of different transformer state signals. The proposed method was applied to real power transformer fault cases, and the research results showed that TSMBE-SCN achieved 99.01%, 99.1%, 99.11%, 99.11%, 99.14% and 99.02% of the diagnostic rates under different folding numbers, respectively, compared with conventional diagnostic models MBE-SCN, TSMSE-SCN, MSE-SCN, TSMDE-SCN and MDE-SCN. This comparison shows that TSMBE-SCN has a strong competitive advantage, which verifies that the proposed method has a good diagnostic effect. This study provides a new method for power transformer fault diagnosis, which has good reference value.     

Sample Entropy as a Tool to Assess Lumbo-Pelvic Movements in a Clinical Test for Low-Back-Pain Patients

Low back pain (LBP) obviously reduces the quality of life but is also the world’s leading cause of years lived with disability. Alterations in motor response and changes in movement patterns are expected in LBP patients when compared to healthy people. Such changes in dynamics may be assessed by the nonlinear analysis of kinematical time series recorded from one patient’s motion. Since sample entropy (SampEn) has emerged as a relevant index measuring the complexity of a given time series, we propose the development of a clinical test based on SampEn of a time series recorded by a wearable inertial measurement unit for repeated bending and returns (b and r) of the trunk. Twenty-three healthy participants were asked to perform, in random order, 50 repetitions of this movement by touching a stool and another 50 repetitions by touching a box on the floor. The angular amplitude of the b and r movement and the sample entropy of the three components of the angular velocity and acceleration were computed. We showed that the repetitive b and r “touch the stool” test could indeed be the basis of a clinical test for the evaluation of low-back-pain patients, with an optimal duration of 70 s, acceptable in daily clinical practice.     

An Entropy Metric for Regular Grammar Classification and Learning with Recurrent Neural Networks

Recently, there has been a resurgence of formal language theory in deep learning research. However, most research focused on the more practical problems of attempting to represent symbolic knowledge by machine learning. In contrast, there has been limited research on exploring the fundamental connection between them. To obtain a better understanding of the internal structures of regular grammars and their corresponding complexity, we focus on categorizing regular grammars by using both theoretical analysis and empirical evidence. Specifically, motivated by the concentric ring representation, we relaxed the original order information and introduced an entropy metric for describing the complexity of different regular grammars. Based on the entropy metric, we categorized regular grammars into three disjoint subclasses: the polynomial, exponential and proportional classes. In addition, several classification theorems are provided for different representations of regular grammars. Our analysis was validated by examining the process of learning grammars with multiple recurrent neural networks. Our results show that as expected more complex grammars are generally more difficult to learn.     

Sound Signal Based Fault Classification System in Motorcycles Using Hybrid Feature Sets and Extreme Learning Machine Classifiers

Vehicles generate dissimilar sound patterns under different working environments. These generated sound patterns signify the condition of the engines, which in turn is used for diagnosing various faults. In this paper, the sound signals produced by motorcycles are analyzed to locate various faults. The important attributes are extracted from the generated sound signals based on time, frequency and wavelet domains which clearly describe the statistical behavior of the signals. Further, various types of faults are classified using the Extreme Learning Machine (ELM) classifier from the extracted features. Moreover, the improved classification performance is obtained by the combination of feature sets in different domains. The simulation results clearly demonstrate that the proposed hybrid feature set together with the ELM classifier gives more promising results with higher classification accuracy when compared with the other conventional methods.     

Assessment of the Effectiveness of Rehabilitation after Total Knee Replacement Surgery Using Sample Entropy and Classical Measures of Body Balance

Exercises in virtual reality (VR) have recently become a popular form of rehabilitation and are reported to be more effective than a standard rehabilitation protocol alone. The aim of this study was to assess the efficacy of adjunct VR training in improving postural control in patients after total knee replacement surgery (TKR). Forty-two patients within 7–14 days of TKR were enrolled and divided into a VR group and a control group (C). The C group underwent standard postoperative rehabilitation. The VR group additionally attended twelve 30-min exercise sessions using the Virtual Balance Clinic prototype system. Balance was assessed on the AMTI plate in bipedal standing with and without visual feedback before and after the four-week rehabilitation. Linear measures and sample entropy of CoP data were analyzed. After four weeks of rehabilitation, a significant reduction in parameters in the sagittal plane and ellipse area was noted while the eyes remained open. Regression analysis showed that sample entropy depended on sex, body weight, visual feedback and age. Based on the sample entropy results, it was concluded that the complexity of the body reaction had not improved. The standing-with-eyes-closed test activates automatic balance mechanisms and offers better possibilities as a diagnostic tool.     

Detecting Unfavorable Driving States in Electroencephalography Based on a PCA Sample Entropy Feature and Multiple Classification Algorithms

Unfavorable driving states can cause a large number of vehicle crashes and are significant factors in leading to traffic accidents. Hence, the aim of this research is to design a robust system to detect unfavorable driving states based on sample entropy feature analysis and multiple classification algorithms. Multi-channel Electroencephalography (EEG) signals are recorded from 16 participants while performing two types of driving tasks. For the purpose of selecting optimal feature sets for classification, principal component analysis (PCA) is adopted for reducing dimensionality of feature sets. Multiple classification algorithms, namely, K nearest neighbor (KNN), decision tree (DT), support vector machine (SVM) and logistic regression (LR) are employed to improve the accuracy of unfavorable driving state detection. We use 10-fold cross-validation to assess the performance of the proposed systems. It is found that the proposed detection system, based on PCA features and the cubic SVM classification algorithm, shows robustness as it obtains the highest accuracy of 97.81%, sensitivity of 96.93%, specificity of 98.73% and precision of 98.75%. Experimental results show that the system we designed can effectively monitor unfavorable driving states.