UAV path design with connectivity constraint based on deep reinforcement learning

Cellular networks are expected to communicate effectively with unmanned aerial vehicles (UAVs) and support various applications. However, existing cellular networks are primarily designed to cover users on the ground; thus, coverage holes in the sky will exist. In this paper, we investigate the problem of path design for cellular-connected UAVs, taking into account the interruption performance throughout the UAV mission to minimize the completion time. Two types of connectivity constraints requirements are assumed to be available. The first is defined as the maximum continuous time interval that the UAV loses connection with base stations (BSs) below a predefined threshold. For the second, we consider the sum outage of UAV is limited during the entire UAV mission. The UAV is tasked with flying from a starting location to a final destination while minimization the mission time, satisfying the two constraints, separately. The formulated path design problem which involves continues variables and a dynamic radio environment, is not convex and thus is extremely difficult to solve directly. To tackle this challenge, a deep reinforcement learning (DRL) based trajectory design algorithm is proposed, where the Dueling Double Deep Q Network(Dueling DDQN) with multi-steps learning method is applied. Simulation results demonstrate the effectiveness of the proposed DRL algorithm and achieve a trade-off between the trajectory length of the UAV and connection quality.     

Relay selection scheme based on deep reinforcement learning in wireless sensor networks

Cooperative communication technology has realized the enhancement in the wireless communication system’s spectrum utilization rate without resorting to any additional equipment; additionally, it ensures system reliability in transmission, increasingly becoming a research focus within the sphere of wireless sensor networks (WSNs). Since the selection of relay is crucial to cooperative communication technology, this paper proposes two different relay selection schemes subject to deep reinforcement learning (DRL), in response to the issues in WSNs with relay selection in cooperative communications, which can be summarized as the Deep-Q-Network Based Relay Selection Scheme (DQN-RSS), as well as the Proximal Policy Optimization Based Relay Selection Scheme (PPO-RSS); it further compared the commonly used Q-learning relay selection scheme (Q-RSS) with random relay selection scheme. First, the cooperative communication process in WSNs is modeled as a Markov decision process, and DRL algorithm is trained in accordance with the outage probability, as well as mutual information (MI). Under the condition of unknown instantaneous channel state information (CSI), the best relay is adaptively selected from multiple candidate relays. Thereafter, in view of the slow convergence speed of Q-RSS in high-dimensional state space, the DRL algorithm is used to accelerate the convergence. In particular, we employ DRL algorithm to deal with high-dimensional state space while speeding up learning. The experimental results reveal that under the same conditions, the random relay selection scheme always has the worst performance. And compared to Q-RSS, the two relay selection schemes designed in this paper greatly reduce the number of iterations and speed up the convergence speed, thereby reducing the computational complexity and overhead of the source node selecting the best relay strategy. In addition, the two relay selection schemes designed and raised in this paper are featured by lower-level outage probability with lower-level energy consumption and larger system capacity. In particular, PPO-RSS has higher reliability and practicability.     

Resource optimization for UAV-assisted mobile edge computing system based on deep reinforcement learning

Computational efficiency is a direction worth considering in moving edge computing (MEC) systems. However, the computational efficiency of UAV-assisted MEC systems is rarely studied. In this paper, we maximize the computational efficiency of the MEC network by optimizing offloading decisions, UAV flight paths, and allocating users’ charging and offloading time reasonably. The method of deep reinforcement learning is used to optimize the resources of UAV-assisted MEC system in complex urban environment, and the user’s computation-intensive tasks are offloaded to the UAV-mounted MEC server, so that the overloaded tasks in the whole system can be alleviated. We study and design a framework algorithm that can quickly adapt to task offload decision making and resource allocation under changing wireless channel conditions in complex urban environments. The optimal offloading decisions from state space to action space is generated through deep reinforcement learning, and then the user’s own charging time and offloading time are rationally allocated to maximize the weighted sum computation rate. Finally, combined with the radio map to optimize the UAC trajectory to improve the overall weighted sum computation rate of the system. Simulation results show that the proposed DRL+TO framework algorithm can significantly improve the weighted sum computation rate of the whole MEC system and save time. It can be seen that the MEC system resource optimization scheme proposed in this paper is feasible and has better performance than other benchmark schemes.     

An Interpretation Architecture for Deep Learning Models with the Application of COVID-19 Diagnosis

The Coronavirus disease 2019 (COVID-19) has become one of the threats to the world. Computed tomography (CT) is an informative tool for the diagnosis of COVID-19 patients. Many deep learning approaches on CT images have been proposed and brought promising performance. However, due to the high complexity and non-transparency of deep models, the explanation of the diagnosis process is challenging, making it hard to evaluate whether such approaches are reliable. In this paper, we propose a visual interpretation architecture for the explanation of the deep learning models and apply the architecture in COVID-19 diagnosis. Our architecture designs a comprehensive interpretation about the deep model from different perspectives, including the training trends, diagnostic performance, learned features, feature extractors, the hidden layers, the support regions for diagnostic decision, and etc. With the interpretation architecture, researchers can make a comparison and explanation about the classification performance, gain insight into what the deep model learned from images, and obtain the supports for diagnostic decisions. Our deep model achieves the diagnostic result of 94.75%, 93.22%, 96.69%, 97.27%, and 91.88% in the criteria of accuracy, sensitivity, specificity, positive predictive value, and negative predictive value, which are 8.30%, 4.32%, 13.33%, 10.25%, and 6.19% higher than that of the compared traditional methods. The visualized features in 2-D and 3-D spaces provide the reasons for the superiority of our deep model. Our interpretation architecture would allow researchers to understand more about how and why deep models work, and can be used as interpretation solutions for any deep learning models based on convolutional neural network. It can also help deep learning methods to take a step forward in the clinical COVID-19 diagnosis field.     

DRL-M4MR: An intelligent multicast routing approach based on DQN deep reinforcement learning in SDN

Traditional multicast routing methods have some problems in constructing a multicast tree. These problems include limited access to network state information, poor adaptability to dynamic and complex changes in the network, and inflexible data forwarding. To address these defects, the optimal multicast routing problem in software-defined networking (SDN) is tailored as a multiobjective optimization problem, and DRL-M4MR, an intelligent multicast routing algorithm based on the deep Q network (DQN) deep reinforcement learning (DRL) method is designed to construct a multicast tree in a software-defined network. First, combining the characteristics of SDN global network-aware information, the multicast tree state matrix, link bandwidth matrix, link delay matrix and link packet loss rate matrix are designed as the state space of the reinforcement learning agent to solve the problem in that the original method cannot make full use of network status information. Second, the action space of the agent is all the links in the network, and the action selection strategy is designed to add the links to the current multicast tree in four cases. Third, single-step and final reward function forms are designed to guide the agent to make decisions to construct the optimal multicast tree. The double network architectures, dueling network architectures and prioritized experience replay are adopted to improve the learning efficiency and convergence of the agent. Finally, after the DRL-M4MR agent is trained, the SDN controller installs the multicast flow entries by reversely traversing the multicast tree to the SDN switches to implement intelligent multicast routing. The experimental results show that, compared with existing algorithms, the multicast tree constructed by DRL-M4MR can obtain better bandwidth, delay, and packet loss rate performance after training, and it can make more intelligent multicast routing decisions in a dynamic network environment. Code and DRL model are available at https://github.com/GuetYe/DRL-M4MR.     

A Transnational and Transregional Study of the Impact and Effectiveness of Social Distancing for COVID-19 Mitigation

We present an analysis of the relationship between SARS-CoV-2 infection rates and a social distancing metric from data for all the states and most populous cities in the United States and Brazil, all the 22 European Economic Community countries and the United Kingdom. We discuss why the infection rate, instead of the effective reproduction number or growth rate of cases, is a proper choice to perform this analysis when considering a wide span of time. We obtain a strong Spearman’s rank order correlation between the social distancing metric and the infection rate in each locality. We show that mask mandates increase the values of Spearman’s correlation in the United States, where a mandate was adopted. We also obtain an explicit numerical relation between the infection rate and the social distancing metric defined in the present work.     

A multi-channel and multi-user dynamic spectrum access algorithm based on deep reinforcement learning in Cognitive Vehicular Networks with sensing error

In this paper, a spectrum access problem is proposed to improve the spectrum access rates of secondary vehicles in Cognitive Vehicular Networks, where the channel capacity mathematic model is established under the conditions of spectrum sensing errors rates and the dynamic occupancy spectrum rates. Meanwhile, an improved Q-learning method is proposed to conform the dynamic communication under the different conditions of the reward functions. In this function, a Deep Q Network method with a modified reward function (IDQN) is proposed to deal with the situation of multi-vehicle in multi-channel. In order to verify the effectiveness of the IDQN method, the Myopic method, the improved Q-learning method, and the traditional DQN method are compared on Python. The simulation results shown that the proposed IDQN method not only outperforms the compared methods in terms of channel utilization and channel capacity but also improves the ability that the vehicle adapts to the dynamic communication environment.     

焊缝超声相控阵检测数据深度学习降噪方法

超声相控阵检测技术在焊缝检测中具有广泛的应用.超声相控阵检测技术检测信号中常混入噪声导致检测成像时难以分辨真实的缺陷特征.这些噪声主要为无关的反射信号和局部相关的结构噪声,传统的超声图像降噪方法难以有效滤除这些噪声,且存在计算效率低、参数优化复杂等问题.该文提出了一种基于深度学习的焊缝超声相控阵检测技术检测S扫图像的降...     

基于多域特征提取和深度学习的声源被动测距*

由于实际海洋环境中存在大量的非高斯噪声,一些基于高斯假设的传统去噪方法在实际海洋环境中性能下降甚至失效。针对非高斯噪声,如α稳定分布噪声、非平稳行船噪声下的脉冲信号的去噪与重构,该文提出一种基于深度学习的方法。去噪模型首先通过学习带噪信号短时傅里叶变换谱与残差谱之间的映射关系以去除环境噪声,之后对去噪信号的时频谱进行逆变换重构脉冲信号。仿真实验结果表明,深度学习模型在非高斯噪声环境下脉冲信号的去噪与重构任务中有着良好的表现,在实测样本上也表现出良好的泛化性,体现了一定的工程应用价值。     

High speed ghost imaging based on a heuristic algorithm and deep learning

We report an overlapping sampling scheme to accelerate computational ghost imaging for imaging moving targets,based on reordering a set of Hadamard modulation matrices by means of a heuristic algorithm. The new condensed overlapped matrices are then designed to shorten and optimize encoding of the overlapped patterns, which are shown to be much superior to the random matrices. In addition, we apply deep learning to image the target, and use the signal acquired by the bucket detector and corresponding real image to train the neural network. Detailed comparisons show that our new method can improve the imaging speed by as much as an order of magnitude, and improve the image quality as well.     

基于深度学习的CSNS加速器预警系统样机

为了能在中国散裂中子源（CSNS）加速器的部分故障发生前发出预警信息,利用深度学习建立了基于CSNS加速器真空度和漂移管直线加速器（DTL）温度的特征模型,开发了一套CSNS加速器预警系统样机。该样机基于实验物理及工业控制系统（EPICS）架构搭建,主要由训练、识别和信息发布3部分组成,采用Python进行程序设计开发,实现了训练样本获取、深度学习网络设计和训练、在线识别和信息发布等功能。测试结果表明,该样机对基于CSNS加速器真空度和DTL温度历史数据生成的测试集的准确率达98.4%,且能根据实时数据识别出CSNS加速器真空度和DTL温度的异常,并能发出预警信息,证明了其可行性和有效性。     

Development of an electronic stopping power model based on deep learning and its application in ion range prediction

Deep learning algorithm emerges as a new method to take the raw features from large dataset and mine their deep implicit relations, which is promising for solving traditional physical challenges. A particularly intricate and difficult challenge is the energy loss mechanism of energetic ions in solid, where accurate prediction of stopping power is a long-time problem. In this work, we develop a deep-learning-based stopping power model with high overall accuracy, and overcome the long-standing deficiency of the existing classical models by improving the predictive accuracy of stopping power for ultra-heavy ion with low energy, and the corresponding projected range. This electronic stopping power model, based on deep learning algorithm, could be hopefully applied for the study of ion-solid interaction mechanism and enormous relevant applications.