To Freeze or Not to Freeze? Epidemic Prevention and Control in the DSGE Model Using an Agent-Based Epidemic Component

The ongoing COVID-19 pandemic has raised numerous questions concerning the shape and range of state interventions the goals of which are to reduce the number of infections and deaths. The lockdowns, which have become the most popular response worldwide, are assessed as being an outdated and economically inefficient way to fight the disease. However, in the absence of efficient cures and vaccines, there is a lack of viable alternatives. In this paper we assess the economic consequences of the epidemic prevention and control schemes that were introduced in order to respond to the COVID-19 pandemic. The analyses report the results of epidemic simulations that were obtained using the agent-based modelling methods under the different response schemes and their use in order to provide conditional forecasts of the standard economic variables. The forecasts were obtained using the dynamic stochastic general equilibrium model (DSGE) with the labour market component.     

SIHR rumor spreading model in social networks

There are significant differences between rumor spreading and epidemic spreading in social networks, especially with consideration of the mutual effect of forgetting and remembering mechanisms. In this paper, a new rumor spreading model, Susceptible-Infected-Hibernator-Removed (SIHR) model, is developed. The model extends the classical Susceptible-Infected-Removed (SIR) rumor spreading model by adding a direct link from ignorants to stiflers and a new kind of people-Hibernators. We derive mean-field equations that describe the dynamics of the SIHR model in social networks. Then a steady-state analysis is conducted to investigate the final size of the rumor spreading under various spreading rate, stifling rate, forgetting rate, and average degree of the network. We discuss the spreading threshold and find the relationship between the final size of the rumor and two probabilities. Also Runge-Kutta method is used for numerical simulation which shows that the direct link from the ignorants to the stiflers advances the rumor terminal time and reduces the maximum rumor influence. Moreover, the forgetting and remembering mechanisms of hibernators postpone the rumor terminal time and reduce the maximum rumor influence.     

Co-Evolution of Predator-Prey Ecosystems by Reinforcement Learning Agents

The problem of finding adequate population models in ecology is important for understanding essential aspects of their dynamic nature. Since analyzing and accurately predicting the intelligent adaptation of multiple species is difficult due to their complex interactions, the study of population dynamics still remains a challenging task in computational biology. In this paper, we use a modern deep reinforcement learning (RL) approach to explore a new avenue for understanding predator-prey ecosystems. Recently, reinforcement learning methods have achieved impressive results in areas, such as games and robotics. RL agents generally focus on building strategies for taking actions in an environment in order to maximize their expected returns. Here we frame the co-evolution of predators and preys in an ecosystem as allowing agents to learn and evolve toward better ones in a manner appropriate for multi-agent reinforcement learning. Recent significant advancements in reinforcement learning allow for new perspectives on these types of ecological issues. Our simulation results show that throughout the scenarios with RL agents, predators can achieve a reasonable level of sustainability, along with their preys.     

Rumor spreading model considering forgetting and remembering mechanisms in inhomogeneous networks

The SIHR rumor spreading model with consideration of the forgetting and remembering mechanisms was studied in homogeneous networks. We further investigate the properties of the SIHR model in inhomogeneous networks. The SIHR model is refined and mean-field equations are derived to describe the dynamics of the rumor spreading model in inhomogeneous networks. Steady-state analysis is carried out, which shows no spreading threshold existing. Numerical simulations are conducted in a BA scale-free network. The simulation results show that the network topology exerts significant influences on the rumor spreading: In comparison with the ER network, the rumor spreads faster and the final size of the rumor is smaller in BA scale-free network; the forgetting and remembering mechanisms greatly impact the final size of the rumor. Finally, through the numerical simulation, we examine the effects that the spreading rate and the stifling rate have on the the influence of the rumor. In addition, the no threshold result is verified.     

A UoI-Optimal Policy for Timely Status Updates with Resource Constraint

Timely status updates are critical in remote control systems such as autonomous driving and the industrial Internet of Things, where timeliness requirements are usually context dependent. Accordingly, the Urgency of Information (UoI) has been proposed beyond the well-known Age of Information (AoI) by further including context-aware weights which indicate whether the monitored process is in an emergency. However, the optimal updating and scheduling strategies in terms of UoI remain open. In this paper, we propose a UoI-optimal updating policy for timely status information with resource constraint. We first formulate the problem in a constrained Markov decision process and prove that the UoI-optimal policy has a threshold structure. When the context-aware weights are known, we propose a numerical method based on linear programming. When the weights are unknown, we further design a reinforcement learning (RL)-based scheduling policy. The simulation reveals that the threshold of the UoI-optimal policy increases as the resource constraint tightens. In addition, the UoI-optimal policy outperforms the AoI-optimal policy in terms of average squared estimation error, and the proposed RL-based updating policy achieves a near-optimal performance without the advanced knowledge of the system model.     

Comparison of immunization strategies in geographical networks

The epidemic spread and immunizations in geographically embedded scale-free (SF) and Watts-Strogatz (WS) networks are numerically investigated. We make a realistic assumption that it takes time which we call the detection time, for a vertex to be identified as infected, and implement two different immunization strategies: one is based on connection neighbors (CN) of the infected vertex with the exact information of the network structure utilized and the other is based on spatial neighbors (SN) with only geographical distances taken into account. We find that the decrease of the detection time is crucial for a successful immunization in general. Simulation results show that for both SF networks and WS networks, the SN strategy always performs better than the CN strategy, especially for more heterogeneous SF networks at long detection time. The observation is verified by checking the number of the infected nodes being immunized. We found that in geographical space, the distance preferences in the network construction process and the geographically decaying infection rate are key factors that make the SN immunization strategy outperforms the CN strategy. It indicates that even in the absence of the full knowledge of network connectivity we can still stop the epidemic spread efficiently only by using geographical information as in the SN strategy, which may have potential applications for preventing the real epidemic spread.     

An identifier-actor-optimizer policy learning architecture for optimal control of continuous-time nonlinear systems

An intelligent solution method is proposed to achieve real-time optimal control for continuous-time nonlinear systems using a novel identifier-actor-optimizer(IAO)policy learning architecture.In this IAO-based policy learning approach,a dynamical identifier is developed to approximate the unknown part of system dynamics using deep neural networks(DNNs).Then,an indirect-method-based optimizer is proposed to generate high-quality optimal actions for system control considering both the constraints and performance index.Furthermore,a DNN-based actor is developed to approximate the obtained optimal actions and return good initial guesses to the optimizer.In this way,the traditional optimal control methods and state-of-the-art DNN techniques are combined in the IAO-based optimal policy learning method.Compared to the reinforcement learning algorithms with actor-critic architectures that suffer hard reward design and low computational efficiency,the IAO-based optimal policy learning algorithm enjoys fewer user-defined parameters,higher learning speeds,and steadier convergence properties in solving complex continuous-time optimal control problems(OCPs).Simulation results of three space flight control missions are given to substantiate the effectiveness of this IAO-based policy learning strategy and to illustrate the performance of the developed DNN-based optimal control method for continuous-time OCPs.     

Epidemic propagation on adaptive coevolutionary networks with preferential local-world reconnecting strategy

In the propagation of an epidemic in a population, individuals adaptively adjust their behavior to avoid the risk of an epidemic. Differently from existing studies where new links are established randomly, a local link is established preferentially in this paper. We propose a new preferentially reconnecting edge strategy depending on spatial distance (PR- SD). For the PR-SD strategy, the new link is established at random with probability p and in a shortest distance with the probability 1 p. We establish the epidemic model on an adaptive network using Cellular Automata, and demonstrate the effectiveness of the proposed model by numerical simulations. The results show that the smaller the value of parameter p, the more difficult the epidemic spread is. The PR-SD strategy breaks long-range links and establishes as many short-range links as possible, which causes the network efficiency to decrease quickly and the propagation of the epidemic is restrained effectively.     

Optimal vaccination and treatment of an epidemic network model

In this Letter, we firstly propose an epidemic network model incorporating two controls which are vaccination and treatment. For the constant controls, by using Lyapunov function, global stability of the disease-free equilibrium and the endemic equilibrium of the model is investigated. For the non-constant controls, by using the optimal control strategy, we discuss an optimal strategy to minimize the total number of the infected and the cost associated with vaccination and treatment. Table 1 and Figs. 1–5 are presented to show the global stability and the efficiency of this optimal control.     

新型冠状病毒肺炎疫情确诊病例的统计分析及自回归建模

基于安徽省卫生健康委员会截至2020年2月19日公布的800余例新型冠状病毒肺炎病例信息,根据病例中公布的接触史构建确诊患者间的有向传播关系,发现源传染患者中男性居多,被传染患者中女性居多.从病例信息中可知,安徽省新型冠状病毒肺炎疫情的发展从初期的具有武汉居住或接触史的输入病例转入后期本地传播为主的小范围社区传播,且严格的防控隔离措施有效切断了社区内的进一步传播.源传染患者与被传染患者的确诊时间间隔可用G分布拟合,确诊时间间隔的中位数为2 d,平均值为2.67 d.基于有向传播关系的统计特点,构建安徽省疫情发展后期的自回归传播模型,模型仿真结果与疫情发展数据符合.对除湖北省的全国确诊病例数据同样采取自回归建模与仿真,结果仍与疫情发展数据符合.这一发现为控制疫情在湖北省以外区域的防控提供了参考:通过严格的防控措施和隔离措施,疫情在湖北省之外的传播具有很大的黏滞性,多为家庭程度的密切接触传播,且能有效控制新型冠状病毒肺炎在当地的传播深度,有效控制了疫情的蔓延.     

Control of chaos in Frenkel–Kontorova model using reinforcement learning

It is shown that we can control spatiotemporal chaos in the Frenkel–Kontorova(FK) model by a model-free control method based on reinforcement learning. The method uses Q-learning to find optimal control strategies based on the reward feedback from the environment that maximizes its performance. The optimal control strategies are recorded in a Q-table and then employed to implement controllers. The advantage of the method is that it does not require an explicit knowledge of the system, target states, and unstable periodic orbits. All that we need is the parameters that we are trying to control and an unknown simulation model that represents the interactive environment. To control the FK model, we employ the perturbation policy on two different kinds of parameters, i.e., the pendulum lengths and the phase angles. We show that both of the two perturbation techniques, i.e., changing the lengths and changing their phase angles, can suppress chaos in the system and make it create the periodic patterns. The form of patterns depends on the initial values of the angular displacements and velocities. In particular, we show that the pinning control strategy, which only changes a small number of lengths or phase angles, can be put into effect.     

Deep transfer reinforcement learning for resource allocation in hybrid multiple access systems

This paper proposes a resource allocation scheme for hybrid multiple access involving both orthogonal multiple access and non-orthogonal multiple access (NOMA) techniques. The proposed resource allocation scheme employs multi-agent deep reinforcement learning (MA-DRL) to maximize the sum-rate for all users. More specifically, the MA-DRL-based scheme jointly allocates subcarrier and power resources for users by utilizing deep Q networks and multi-agent deep deterministic policy gradient networks. Meanwhile, an adaptive learning determiner mechanism is introduced into our allocation scheme to achieve better sum-rate performance. However, the above deep reinforcement learning adopted by our scheme cannot optimize parameters quickly in the new communication model. In order to better adapt to the new environment and make the resource allocation strategy more robust, we propose a transfer learning scheme based on deep reinforcement learning (T-DRL). The T-DRL-based scheme allows us to transfer the subcarrier allocation network and the power allocation network collectively or independently. Simulation results show that the proposed MA-DRL-based resource allocation scheme can achieve better sum-rate performance. Furthermore, the T-DRL-based scheme can effectively improve the convergence speed of the deep resource allocation network.     

Prediction of epidemics dynamics on networks with partial differential equations: A case study for COVID-19 in China

Since December 2019, the COVID-19 epidemic has repeatedly hit countries around the world due to various factors such as trade, national policies and the natural environment. To closely monitor the emergence of new COVID-19 clusters and ensure high prediction accuracy, we develop a new prediction framework for studying the spread of epidemic on networks based on partial differential equations (PDEs), which captures epidemic diffusion along the edges of a network driven by population flow data. In this paper, we focus on the effect of the population movement on the spread of COVID-19 in several cities from different geographic regions in China for describing the transmission characteristics of COVID-19. Experiment results show that the PDE model obtains relatively good prediction results compared with several typical mathematical models. Furthermore, we study the effectiveness of intervention measures, such as traffic lockdowns and social distancing, which provides a new approach for quantifying the effectiveness of the government policies toward controlling COVID-19 via the adaptive parameters of the model. To our knowledge, this work is the first attempt to apply the PDE model on networks with Baidu Migration Data for COVID-19 prediction.     

A dynamic epidemic control model on uncorrelated complex networks

In this paper, a dynamic epidemic control model on the uncorrelated complex networks is proposed. By means of theoretical analysis, we found that the new model has a similar epidemic threshold as that of the susceptible-infectedrecovered （SIR） model on the above networks, but it can reduce the prevalence of the infected individuals remarkably. This result may help us understand epidemic spreading phenomena on real networks and design appropriate strategies to control infections.     

Dynamics Analysis of a Wireless Rechargeable Sensor Network for Virus Mutation Spreading

Virus spreading problems in wireless rechargeable sensor networks (WSNs) are becoming a hot topic, and the problem has been studied and discussed in recent years. Many epidemic spreading models have been introduced for revealing how a virus spreads and how a virus is suppressed. However, most of them assumed the sensors are not rechargeable sensors. In addition, most of existing works do not consider virus mutation problems. This paper proposes a novel epidemic model, including susceptible, infected, variant, low-energy and dead states, which considers the rechargeable sensors and the virus mutation factor. The stability of the proposed model is first analyzed by adopting the characteristic equation and constructing Lyapunov functions methods. Then, an optimal control problem is formulated to control the virus spread and decrease the cost of the networks by applying Pontryagin’s maximum principle. Finally, all of the theoretical results are confirmed by numerical simulation.     

Influenza epidemic spread simulation for Poland — a large scale, individual based model study

In this work a construction of an agent based model for studying the effects of influenza epidemic in large scale (38 million individuals) stochastic simulations, together with the resulting various scenarios of disease spread in Poland are reported. Simple transportation rules were employed to mimic individuals’ travels in dynamic route-changing schemes, allowing for the infection spread during a journey. Parameter space was checked for stable behaviour, especially towards the effective infection transmission rate variability. Although the model reported here is based on quite simple assumptions, it allowed to observe two different types of epidemic scenarios: characteristic for urban and rural areas. This differentiates it from the results obtained in the analogous studies for the UK or US, where settlement and daily commuting patterns are both substantially different and more diverse. The resulting epidemic scenarios from these ABM simulations were compared with simple, differential equations based, SIR models — both types of the results displaying strong similarities. The pDYN software platform developed here is currently used in the next stage of the project employed to study various epidemic mitigation strategies.     

Scheduling Strategy Design Framework for Cyber–Physical System with Non-Negligible Propagation Delay

Cyber–physical systems (CPS) have been widely employed as wireless control networks. There is a special type of CPS which is developed from the wireless networked control systems (WNCS). They usually include two communication links: Uplink transmission and downlink transmission. Those two links form a closed-loop. When such CPS are deployed for time-sensitive applications such as remote control, the uplink and downlink propagation delay are non-negligible. However, existing studies on CPS/WNCS usually ignore the propagation delay of the uplink and downlink channels. In order to achieve the best balance between uplink and downlink transmissions under such circumstances, we propose a heuristic framework to obtain the optimal scheduling strategy that can minimize the long-term average control cost. We model the optimization problem as a Markov decision process (MDP), and then give the sufficient conditions for the existence of the optimal scheduling strategy. We propose the semi-predictive framework to eliminate the impact of the coupling characteristic between the uplink and downlink data packets. Then we obtain the lookup table-based optimal offline strategy and the neural network-based suboptimal online strategy. Numerical simulation shows that the scheduling strategies obtained by this framework can bring significant performance improvements over the existing strategies.     

Evolutionary Dynamics of Resource Allocation in the Colonel Blotto Game

We provide an evolutionary game-theoretical formulation for a model of resource allocation—the Colonel Blotto game. In this game, two players with different total resources must entirely distribute them among a set of items. Each item is won by the player that assigned higher resources to it, and the payoff of each player is the total number of won items. Our evolutionary formulation makes it possible to obtain optimal strategies as the equilibrium states of a dynamical process. At the same time, it naturally requires considering a population of players—whose strategies evolve by imitation and random fluctuations—thus better approaching a realistic situation with many economic agents. Results show, in particular, how agents with low total resources manage to maximize their winnings in spite of their intrinsically disadvantageous condition.     

The Value of Information Searching against Fake News

Inspired by the Daley-Kendall and Goffman-Newill models, we propose an Ignorant-Believer-Unbeliever rumor (or fake news) spreading model with the following characteristics: (i) a network contact between individuals that determines the spread of rumors; (ii) the value (cost versus benefit) for individuals who search for truthful information (learning); (iii) an impact measure that assesses the risk of believing the rumor; (iv) an individual search strategy based on the probability that an individual searches for truthful information; (v) the population search strategy based on the proportion of individuals of the population who decide to search for truthful information; (vi) a payoff for the individuals that depends on the parameters of the model and the strategies of the individuals. Furthermore, we introduce evolutionary information search dynamics and study the dynamics of population search strategies. For each value of searching for information, we compute evolutionarily stable information (ESI) search strategies (occurring in non-cooperative environments), which are the attractors of the information search dynamics, and the optimal information (OI) search strategy (occurring in (eventually forced) cooperative environments) that maximizes the expected information payoff for the population. For rumors that are advantageous or harmful to the population (positive or negative impact), we show the existence of distinct scenarios that depend on the value of searching for truthful information. We fully discuss which evolutionarily stable information (ESI) search strategies and which optimal information (OI) search strategies eradicate (or not) the rumor and the corresponding expected payoffs. As a corollary of our results, a recommendation for legislators and policymakers who aim to eradicate harmful rumors is to make the search for truthful information free or rewarding.     

Simplified model on the timing of easing the lockdown

Lockdown procedures have been proven successful in mitigating the spread of the viruses in this COVID-19 pandemic, but they also have devastating impact on the economy. We use a modified Susceptible-Infectious-Recovered-Deceased model with time dependent infection rate to simulate how the infection is spread under lockdown. The economic cost due to the loss of workforce and incurred medical expenses is evaluated with a simple model. We find the best strategy, meaning the smallest economic cost for the entire course of the pandemic, is to keep the strict lockdown as long as possible.