Schedule-Based Cooperative Multi-agent Reinforcement Learning for Multi-channel Communication in Wireless Sensor Networks

Wireless sensor networks (WSNs) have become an important component in the Internet of things (IoT) field. In WSNs, multi-channel protocols have been developed to overcome some limitations related to the throughput and delivery rate which have become necessary for many IoT applications that require sufficient bandwidth to transmit a large amount of data. However, the requirement of frequent negotiation for channel assignment in distributed multi-channel protocols incurs an extra-large communication overhead which results in a reduction of the network lifetime. To deal with this requirement in an energy-efficient way is a challenging task. Hence, the Reinforcement Learning (RL) approach for channel assignment is used to overcome this problem. Nevertheless, the use of the RL approach requires a number of iterations to obtain the best solution which in turn creates a communication overhead and time-wasting. In this paper, a Self-schedule based Cooperative multi-agent Reinforcement Learning for Channel Assignment (SCRL CA) approach is proposed to improve the network lifetime and performance. The proposal addresses both regular traffic scheduling and assignment of the available orthogonal channels in an energy-efficient way. We solve the cooperation between the RL agents problem by using the self-schedule method to accelerate the RL iterations, reduce the communication overhead and balance the energy consumption in the route selection process. Therefore, two algorithms are proposed, the first one is for the Static channel assignment (SSCRL CA) while the second one is for the Dynamic channel assignment (DSCRL CA). The results of extensive simulation experiments show the effectiveness of our approach in improving the network lifetime and performance through the two algorithms.

     

A new code construction with zero cross correlation based on BIBD

This paper presents a newly constructed zero cross correlation code (ZCC) which is based on BIBD (balanced incomplete block design) code. The ZCC (C, w) code is a family of binary sequences of length C and constant Hamming-weight w. Such codes find applications in spectral amplitude-coding optical code division multiple access (SAC-OCDMA). The constructing ZCC codes have a size of C ? N ÿ w + 1, where N is the number of users and C is any prime number. The proposed construction method is not complicated compared to the existing ones.     

Multi-dimensional Capon spectral estimation using discrete Zhang neural networks

The minimum variance spectral estimator, also known as the Capon spectral estimator, is a high resolution spectral estimator used extensively in practice. In this paper, we derive a novel implementation of a very computationally demanding matched filter-bank based a spectral estimator, namely the multi-dimensional Capon spectral estimator. To avoid the direct computation of the inverse covariance matrix used to estimate the Capon spectrum which can be computationally very expensive, particularly when the dimension of the matrix is large, we propose to use the discrete Zhang neural network for the online covariance matrix inversion. The computational complexity of the proposed algorithm for one-dimensional (1-D), as well as for two-dimensional (2-D) and three-dimensional (3-D) data sequences is lower when a parallel implementation is used.     

Design of Biorthogonal Filter Banks Using a Multi-objective Genetic Algorithm for an Image Coding Scheme

In this paper, we present an optimization method based on a multi-objective Genetic Algorithm (GA) for the design of linear phase filter banks for an image coding scheme. To be effective, the filter banks should satisfy a number of desirable criteria related to such a scheme. Instead of imposing the entire PR condition as in conventional designs, we introduce flexibility in the design by relaxing the Perfect Reconstruction (PR) condition and defining a PR violation measure as an objective criterion to maintain near perfect reconstruction (N-PR) filter banks. Particularly in this work, the designed filter banks are near-orthogonal. This has been made possible by minimizing the deviation from the orthogonality in the optimization process. The optimization problem is formulated as a constrained multi-objective, and a modified Nondominated Sorting Genetic Algorithm NSGAII is proposed in this work to find the Pareto optimal solutions that achieve the best compromise between the different objective criteria. The experimental results show that the filter banks designed with the proposed method outperform significantly the 9/7 filter bank of JPEG2000 in most cases. Furthermore, the filter banks are near orthogonal. This is very helpful, especially where embedded coding is required.     

QRS complex detection based on multi wavelet packet decomposition

We present in this paper a wavelet packet based QRS complex detection algorithm. Our proposed algorithm consists of a particular combination of two vectors obtained by applying a designed routine of QRS detection process using ‘haar’ and ‘db10’ wavelet functions respectively. The QRS complex detection routine is based on the histogram approach where our key idea was to search for the node with highest number of histogram coefficients, at center, which we assume that they are related to the iso-electric baseline whereas the remaining least number coefficients reflect the R waves peaks. Following a classical approach based of a calculated fixed threshold, the possible QRS complexes will be determined. The QRS detection complex algorithm has been applied to the whole MIT-BIH arrhythmia Database to assess its robustness. The algorithm reported a global sensitivity of 98.68%, positive predictive value of 97.24% and a percentage error of 04.12%. Eventhough, the obtained global results are not as excellent as expected, we have demonstrate that our designed QRS detection algorithm performs good on a partial selected high percentage of the whole database, e.g., the partial results, obtained when applying the algorithm on 85.01% of the whole MIT-BIH arrhythmia Database, are 99.14% of sensitivity, 98.94% of positive predictive value and 01.92% of percentage error.