Boosting COVID-19 Image Classification Using MobileNetV3 and Aquila Optimizer Algorithm

Currently, the world is still facing a COVID-19 (coronavirus disease 2019) classified as a highly infectious disease due to its rapid spreading. The shortage of X-ray machines may lead to critical situations and delay the diagnosis results, increasing the number of deaths. Therefore, the exploitation of deep learning (DL) and optimization algorithms can be advantageous in early diagnosis and COVID-19 detection. In this paper, we propose a framework for COVID-19 images classification using hybridization of DL and swarm-based algorithms. The MobileNetV3 is used as a backbone feature extraction to learn and extract relevant image representations as a DL model. As a swarm-based algorithm, the Aquila Optimizer (Aqu) is used as a feature selector to reduce the dimensionality of the image representations and improve the classification accuracy using only the most essential selected features. To validate the proposed framework, two datasets with X-ray and CT COVID-19 images are used. The obtained results from the experiments show a good performance of the proposed framework in terms of classification accuracy and dimensionality reduction during the feature extraction and selection phases. The Aqu feature selection algorithm achieves accuracy better than other methods in terms of performance metrics.     

Benchmarking Attention-Based Interpretability of Deep Learning in Multivariate Time Series Predictions

The adaptation of deep learning models within safety-critical systems cannot rely only on good prediction performance but needs to provide interpretable and robust explanations for their decisions. When modeling complex sequences, attention mechanisms are regarded as the established approach to support deep neural networks with intrinsic interpretability. This paper focuses on the emerging trend of specifically designing diagnostic datasets for understanding the inner workings of attention mechanism based deep learning models for multivariate forecasting tasks. We design a novel benchmark of synthetically designed datasets with the transparent underlying generating process of multiple time series interactions with increasing complexity. The benchmark enables empirical evaluation of the performance of attention based deep neural networks in three different aspects: (i) prediction performance score, (ii) interpretability correctness, (iii) sensitivity analysis. Our analysis shows that although most models have satisfying and stable prediction performance results, they often fail to give correct interpretability. The only model with both a satisfying performance score and correct interpretability is IMV-LSTM, capturing both autocorrelations and crosscorrelations between multiple time series. Interestingly, while evaluating IMV-LSTM on simulated data from statistical and mechanistic models, the correctness of interpretability increases with more complex datasets.     

Occlusion-Based Explanations in Deep Recurrent Models for Biomedical Signals

The biomedical field is characterized by an ever-increasing production of sequential data, which often come in the form of biosignals capturing the time-evolution of physiological processes, such as blood pressure and brain activity. This has motivated a large body of research dealing with the development of machine learning techniques for the predictive analysis of such biosignals. Unfortunately, in high-stakes decision making, such as clinical diagnosis, the opacity of machine learning models becomes a crucial aspect to be addressed in order to increase the trust and adoption of AI technology. In this paper, we propose a model agnostic explanation method, based on occlusion, that enables the learning of the input’s influence on the model predictions. We specifically target problems involving the predictive analysis of time-series data and the models that are typically used to deal with data of such nature, i.e., recurrent neural networks. Our approach is able to provide two different kinds of explanations: one suitable for technical experts, who need to verify the quality and correctness of machine learning models, and one suited to physicians, who need to understand the rationale underlying the prediction to make aware decisions. A wide experimentation on different physiological data demonstrates the effectiveness of our approach both in classification and regression tasks.     

Federated Quantum Machine Learning

Distributed training across several quantum computers could significantly improve the training time and if we could share the learned model, not the data, it could potentially improve the data privacy as the training would happen where the data is located. One of the potential schemes to achieve this property is the federated learning (FL), which consists of several clients or local nodes learning on their own data and a central node to aggregate the models collected from those local nodes. However, to the best of our knowledge, no work has been done in quantum machine learning (QML) in federation setting yet. In this work, we present the federated training on hybrid quantum-classical machine learning models although our framework could be generalized to pure quantum machine learning model. Specifically, we consider the quantum neural network (QNN) coupled with classical pre-trained convolutional model. Our distributed federated learning scheme demonstrated almost the same level of trained model accuracies and yet significantly faster distributed training. It demonstrates a promising future research direction for scaling and privacy aspects.     

Co-Training for Visual Object Recognition Based on Self-Supervised Models Using a Cross-Entropy Regularization

Automatic recognition of visual objects using a deep learning approach has been successfully applied to multiple areas. However, deep learning techniques require a large amount of labeled data, which is usually expensive to obtain. An alternative is to use semi-supervised models, such as co-training, where multiple complementary views are combined using a small amount of labeled data. A simple way to associate views to visual objects is through the application of a degree of rotation or a type of filter. In this work, we propose a co-training model for visual object recognition using deep neural networks by adding layers of self-supervised neural networks as intermediate inputs to the views, where the views are diversified through the cross-entropy regularization of their outputs. Since the model merges the concepts of co-training and self-supervised learning by considering the differentiation of outputs, we called it Differential Self-Supervised Co-Training (DSSCo-Training). This paper presents some experiments using the DSSCo-Training model to well-known image datasets such as MNIST, CIFAR-100, and SVHN. The results indicate that the proposed model is competitive with the state-of-art models and shows an average relative improvement of 5% in accuracy for several datasets, despite its greater simplicity with respect to more recent approaches.     

Total aromatics of diesel fuels analysis by deep learning and near-infrared spectroscopy

Abstract

This article proposes a method for quickly detecting the total aromatics content of diesel fuels based on a deep learning algorithm and near-infrared spectroscopy. First, extract the features of the diesel fuel spectral data and eliminate redundant information through the deep belief nets. Second, use a new extreme learning machine to build a prediction model. The experimental results show that the root-mean-square error of prediction value is 0.715 and the coefficient of determination value is 0.988, which proves that the method has good performance. Compared with other typical prediction methods, this method has better predictive ability.     

Development of an electronic stopping power model based on deep learning and its application in ion range prediction

Deep learning algorithm emerges as a new method to take the raw features from large dataset and mine their deep implicit relations, which is promising for solving traditional physical challenges. A particularly intricate and difficult challenge is the energy loss mechanism of energetic ions in solid, where accurate prediction of stopping power is a long-time problem. In this work, we develop a deep-learning-based stopping power model with high overall accuracy, and overcome the long-standing deficiency of the existing classical models by improving the predictive accuracy of stopping power for ultra-heavy ion with low energy, and the corresponding projected range. This electronic stopping power model, based on deep learning algorithm, could be hopefully applied for the study of ion-solid interaction mechanism and enormous relevant applications.     

Ensemble Neuroevolution-Based Approach for Multivariate Time Series Anomaly Detection

Multivariate time series anomaly detection is a widespread problem in the field of failure prevention. Fast prevention means lower repair costs and losses. The amount of sensors in novel industry systems makes the anomaly detection process quite difficult for humans. Algorithms that automate the process of detecting anomalies are crucial in modern failure prevention systems. Therefore, many machine learning models have been designed to address this problem. Mostly, they are autoencoder-based architectures with some generative adversarial elements. This work shows a framework that incorporates neuroevolution methods to boost the anomaly detection scores of new and already known models. The presented approach adapts evolution strategies for evolving an ensemble model, in which every single model works on a subgroup of data sensors. The next goal of neuroevolution is to optimize the architecture and hyperparameters such as the window size, the number of layers, and the layer depths. The proposed framework shows that it is possible to boost most anomaly detection deep learning models in a reasonable time and a fully automated mode. We ran tests on the SWAT and WADI datasets. To the best of our knowledge, this is the first approach in which an ensemble deep learning anomaly detection model is built in a fully automatic way using a neuroevolution strategy.     

Optical Learning Neural Network Using a Selfoc Microlens Array for Pattern Recognition

An optical system for learning neural networks with a 2-D architecture was constructed using a Selfoc microlens array. Using this system, we achieved pattern recognition of typed alphabet characters detected directly with a CCD camera. The system learned 4 characters according to a random search algorithm in order to avoid the difficulties and the costs of calculations of learning signals, optical alignments and addressing to the device which display the weight tensors.     

Conditional Deep Gaussian Processes: Empirical Bayes Hyperdata Learning

It is desirable to combine the expressive power of deep learning with Gaussian Process (GP) in one expressive Bayesian learning model. Deep kernel learning showed success as a deep network used for feature extraction. Then, a GP was used as the function model. Recently, it was suggested that, albeit training with marginal likelihood, the deterministic nature of a feature extractor might lead to overfitting, and replacement with a Bayesian network seemed to cure it. Here, we propose the conditional deep Gaussian process (DGP) in which the intermediate GPs in hierarchical composition are supported by the hyperdata and the exposed GP remains zero mean. Motivated by the inducing points in sparse GP, the hyperdata also play the role of function supports, but are hyperparameters rather than random variables. It follows our previous moment matching approach to approximate the marginal prior for conditional DGP with a GP carrying an effective kernel. Thus, as in empirical Bayes, the hyperdata are learned by optimizing the approximate marginal likelihood which implicitly depends on the hyperdata via the kernel. We show the equivalence with the deep kernel learning in the limit of dense hyperdata in latent space. However, the conditional DGP and the corresponding approximate inference enjoy the benefit of being more Bayesian than deep kernel learning. Preliminary extrapolation results demonstrate expressive power from the depth of hierarchy by exploiting the exact covariance and hyperdata learning, in comparison with GP kernel composition, DGP variational inference and deep kernel learning. We also address the non-Gaussian aspect of our model as well as way of upgrading to a full Bayes inference.     

An Improved Residual Network for Pork Freshness Detection Using Near-Infrared Spectroscopy

Effective and rapid assessment of pork freshness is significant for monitoring pork quality. However, a traditional sensory evaluation method is subjective and physicochemical analysis is time-consuming. In this study, the near-infrared spectroscopy (NIRS) technique, a fast and non-destructive analysis method, is employed to determine pork freshness. Considering that commonly used statistical modeling methods require preprocessing data for satisfactory performance, this paper presents a one-dimensional squeeze-and-excitation residual network (1D-SE-ResNet) to construct the complex relationship between pork freshness and NIRS. The developed model enhances the one-dimensional residual network (1D-ResNet) with squeeze-and-excitation (SE) blocks. As a deep learning model, the proposed method is capable of extracting features from the input spectra automatically and can be used as an end-to-end model to simplify the modeling process. A comparison between the proposed method and five popular classification models indicates that the 1D-SE-ResNet achieves the best performance, with a classification accuracy of 93.72%. The research demonstrates that the NIRS analysis technique based on deep learning provides a promising tool for pork freshness detection and therefore is helpful for ensuring food safety.     

Probabilistic Predictions with Federated Learning

Probabilistic predictions with machine learning are important in many applications. These are commonly done with Bayesian learning algorithms. However, Bayesian learning methods are computationally expensive in comparison with non-Bayesian methods. Furthermore, the data used to train these algorithms are often distributed over a large group of end devices. Federated learning can be applied in this setting in a communication-efficient and privacy-preserving manner but does not include predictive uncertainty. To represent predictive uncertainty in federated learning, our suggestion is to introduce uncertainty in the aggregation step of the algorithm by treating the set of local weights as a posterior distribution for the weights of the global model. We compare our approach to state-of-the-art Bayesian and non-Bayesian probabilistic learning algorithms. By applying proper scoring rules to evaluate the predictive distributions, we show that our approach can achieve similar performance as the benchmark would achieve in a non-distributed setting.     

Deep Task-Based Quantization

Quantizers play a critical role in digital signal processing systems. Recent works have shown that the performance of acquiring multiple analog signals using scalar analog-to-digital converters (ADCs) can be significantly improved by processing the signals prior to quantization. However, the design of such hybrid quantizers is quite complex, and their implementation requires complete knowledge of the statistical model of the analog signal. In this work we design data-driven task-oriented quantization systems with scalar ADCs, which determine their analog-to-digital mapping using deep learning tools. These mappings are designed to facilitate the task of recovering underlying information from the quantized signals. By using deep learning, we circumvent the need to explicitly recover the system model and to find the proper quantization rule for it. Our main target application is multiple-input multiple-output (MIMO) communication receivers, which simultaneously acquire a set of analog signals, and are commonly subject to constraints on the number of bits. Our results indicate that, in a MIMO channel estimation setup, the proposed deep task-bask quantizer is capable of approaching the optimal performance limits dictated by indirect rate-distortion theory, achievable using vector quantizers and requiring complete knowledge of the underlying statistical model. Furthermore, for a symbol detection scenario, it is demonstrated that the proposed approach can realize reliable bit-efficient hybrid MIMO receivers capable of setting their quantization rule in light of the task.     

Personalized Image Classification by Semantic Embedding and Active Learning

Currently, deep learning has shown state-of-the-art performance in image classification with pre-defined taxonomy. However, in a more real-world scenario, different users usually have different classification intents given an image collection. To satisfactorily personalize the requirement, we propose an interactive image classification system with an offline representation learning stage and an online classification stage. During the offline stage, we learn a deep model to extract the feature with higher flexibility and scalability for different users’ preferences. Instead of training the model only with the inter-class discrimination, we also encode the similarity between the semantic-embedding vectors of the category labels into the model. This makes the extracted feature adapt to multiple taxonomies with different granularities. During the online session, an annotation task iteratively alternates with a high-throughput verification task. When performing the verification task, the users are only required to indicate the incorrect prediction without giving the exact category label. For each iteration, our system chooses the images to be annotated or verified based on interactive efficiency optimization. To provide a high interactive rate, a unified active learning algorithm is used to search the optimal annotation and verification set by minimizing the expected time cost. After interactive annotation and verification, the new classified images are used to train a customized classifier online, which reflects the user-adaptive intent of categorization. The learned classifier is then used for subsequent annotation and verification tasks. Experimental results under several public image datasets show that our method outperforms existing methods.     

Prediction carbon dioxide solubility in ionic liquids based on deep learning

Ionic liquids have a great potential in capture and separation of carbon dioxide (CO₂), and the solubility of CO₂ in ionic liquids is one of key data for engineering applications. In this paper, the critical properties of ionic liquids are combined with deep learning models (CP-DNN, CP-CNN, CP-RNN) to establish theoretical prediction models of CO₂ solubility in ionic liquids. The predictive performance of these framworks is able to meet or exceed the predicted effects of the method based on thermodynamic models (PR,SRK) and machine learning method (XGBoost). For CP-RNN, the coefficient of determination (R²) between experimental and predicted values is 0.988, CP-CNN is 0.999, and CP-DNN is 0.984. This research can avoid complex computational characterisation, it is to provide a theoretical method to further enrich and improve the data information analysis of the solubility of CO₂ in ionic liquids.     

A data-efficient self-supervised deep learning model for design and characterization of nanophotonic structures

With its tremendous success in many machine learning and pattern recognition tasks, deep learning, as one type of data-driven models, has also led to many breakthroughs in other disciplines including physics, chemistry and material science. Nevertheless,the supremacy of deep learning over conventional optimization approaches heavily depends on the huge amount of data collected in advance to train the model, which is a common bottleneck of such a data-driven technique. In this work, we present a comprehensive deep learning model for the design and characterization of nanophotonic structures, where a self-supervised learning mechanism is introduced to alleviate the burden of data acquisition. Taking reflective metasurfaces as an example, we demonstrate that the self-supervised deep learning model can effectively utilize randomly generated unlabeled data during training, with the total test loss and prediction accuracy improved by about 15% compared with the fully supervised counterpart.The proposed self-supervised learning scheme provides an efficient solution for deep learning models in some physics-related tasks where labeled data are limited or expensive to collect.     

An Appraisal of Incremental Learning Methods

As a special case of machine learning, incremental learning can acquire useful knowledge from incoming data continuously while it does not need to access the original data. It is expected to have the ability of memorization and it is regarded as one of the ultimate goals of artificial intelligence technology. However, incremental learning remains a long term challenge. Modern deep neural network models achieve outstanding performance on stationary data distributions with batch training. This restriction leads to catastrophic forgetting for incremental learning scenarios since the distribution of incoming data is unknown and has a highly different probability from the old data. Therefore, a model must be both plastic to acquire new knowledge and stable to consolidate existing knowledge. This review aims to draw a systematic review of the state of the art of incremental learning methods. Published reports are selected from Web of Science, IEEEXplore, and DBLP databases up to May 2020. Each paper is reviewed according to the types: architectural strategy, regularization strategy and rehearsal and pseudo-rehearsal strategy. We compare and discuss different methods. Moreover, the development trend and research focus are given. It is concluded that incremental learning is still a hot research area and will be for a long period. More attention should be paid to the exploration of both biological systems and computational models.     

Deep transfer reinforcement learning for resource allocation in hybrid multiple access systems

This paper proposes a resource allocation scheme for hybrid multiple access involving both orthogonal multiple access and non-orthogonal multiple access (NOMA) techniques. The proposed resource allocation scheme employs multi-agent deep reinforcement learning (MA-DRL) to maximize the sum-rate for all users. More specifically, the MA-DRL-based scheme jointly allocates subcarrier and power resources for users by utilizing deep Q networks and multi-agent deep deterministic policy gradient networks. Meanwhile, an adaptive learning determiner mechanism is introduced into our allocation scheme to achieve better sum-rate performance. However, the above deep reinforcement learning adopted by our scheme cannot optimize parameters quickly in the new communication model. In order to better adapt to the new environment and make the resource allocation strategy more robust, we propose a transfer learning scheme based on deep reinforcement learning (T-DRL). The T-DRL-based scheme allows us to transfer the subcarrier allocation network and the power allocation network collectively or independently. Simulation results show that the proposed MA-DRL-based resource allocation scheme can achieve better sum-rate performance. Furthermore, the T-DRL-based scheme can effectively improve the convergence speed of the deep resource allocation network.     

Rhythm Analysis during Cardiopulmonary Resuscitation Using Convolutional Neural Networks

Chest compressions during cardiopulmonary resuscitation (CPR) induce artifacts in the ECG that may provoque inaccurate rhythm classification by the algorithm of the defibrillator. The objective of this study was to design an algorithm to produce reliable shock/no-shock decisions during CPR using convolutional neural networks (CNN). A total of 3319 ECG segments of 9 s extracted during chest compressions were used, whereof 586 were shockable and 2733 nonshockable. Chest compression artifacts were removed using a Recursive Least Squares (RLS) filter, and the filtered ECG was fed to a CNN classifier with three convolutional blocks and two fully connected layers for the shock/no-shock classification. A 5-fold cross validation architecture was adopted to train/test the algorithm, and the proccess was repeated 100 times to statistically characterize the performance. The proposed architecture was compared to the most accurate algorithms that include handcrafted ECG features and a random forest classifier (baseline model). The median (90% confidence interval) sensitivity, specificity, accuracy and balanced accuracy of the method were 95.8% (94.6–96.8), 96.1% (95.8–96.5), 96.1% (95.7–96.4) and 96.0% (95.5–96.5), respectively. The proposed algorithm outperformed the baseline model by 0.6-points in accuracy. This new approach shows the potential of deep learning methods to provide reliable diagnosis of the cardiac rhythm without interrupting chest compression therapy.