Discovering Latent Variables for the Tasks With Confounders in Multi-Agent Reinforcement Learning

Efficient exploration in complex coordination tasks has been considered a challenging problem in multi-agent reinforcement learning(MARL). It is significantly more difficult for those tasks with latent variables that agents cannot directly observe. However, most of the existing latent variable discovery methods lack a clear representation of latent variables and an effective evaluation of the influence of latent variables on the agent. In this paper, we propose a new MARL algorithm based on the soft actor-critic method for complex continuous control tasks with confounders. It is called the multi-agent soft actor-critic with latent variable(MASAC-LV) algorithm, which uses variational inference theory to infer the compact latent variables representation space from a large amount of offline experience.Besides, we derive the counterfactual policy whose input has no latent variables and quantify the difference between the actual policy and the counterfactual policy via a distance function. This quantified difference is considered an intrinsic motivation that gives additional rewards based on how much the latent variable affects each agent. The proposed algorithm is evaluated on two collaboration tasks with confounders, and the experimental results demonstrate the effectiveness of MASAC-LV compared to other baseline algorithms.

Concept and Research Framework for Coordinated Situation Awareness and Active Defense of Cyber-physical Power Systems Against Cyber-attacks

Due to the tight coupling between the cyber and physical sides of a cyber-physical power system(CPPS),the safe and reliable operation of CPPSs is being increasingly impacted by cyber security.This situation poses a challenge to traditional security defense systems,which considers the threat from only one side,i.e.,cyber or physical.To cope with cyberattacks,this paper reaches beyond the traditional one-side security defense systems and proposes the concept of cyber-physical coordinated situation awareness and active defense to improve the ability of CPPSs.An example of a regional frequency control system is used to show the validness and potential of this concept.Then,the research framework is presented for studying and implementing this concept.Finally,key technologies for cyber-physical coordinated situation awareness and active defense against cyber-attacks are introduced.

Detection and Defense Method Against False Data Injection Attacks for Distributed Load Frequency Control System in Microgrid

In the realm of microgrid(MG),the distributed load frequency control(LFC)system has proven to be highly susceptible to the negative effects of false data injection attacks(FDIAs).Considering the significant responsibility of the distributed LFC system for maintaining frequency stability within the MG,this paper proposes a detection and defense method against unobservable FDIAs in the distributed LFC system.Firstly,the method integrates a bi-directional long short-term memory(Bi LSTM)neural network and an improved whale optimization algorithm(IWOA)into the LFC controller to detect and counteract FDIAs.Secondly,to enable the Bi LSTM neural network to proficiently detect multiple types of FDIAs with utmost precision,the model employs a historical MG dataset comprising the frequency and power variances.Finally,the IWOA is utilized to optimize the proportional-integral-derivative(PID)controller parameters to counteract the negative impacts of FDIAs.The proposed detection and defense method is validated by building the distributed LFC system in Simulink.

Efficient Exploration for Multi-Agent Reinforcement Learning via Transferable Successor Features

In multi-agent reinforcement learning(MARL),the behaviors of each agent can influence the learning of others,and the agents have to search in an exponentially enlarged joint-action space.Hence,it is challenging for the multi-agent teams to explore in the environment.Agents may achieve suboptimal policies and fail to solve some complex tasks.To improve the exploring efficiency as well as the performance of MARL tasks,in this paper,we propose a new approach by transferring the knowledge across tasks.Differently from the traditional MARL algorithms,we first assume that the reward functions can be computed by linear combinations of a shared feature function and a set of taskspecific weights.Then,we define a set of basic MARL tasks in the source domain and pre-train them as the basic knowledge for further use.Finally,once the weights for target tasks are available,it will be easier to get a well-performed policy to explore in the target domain.Hence,the learning process of agents for target tasks is speeded up by taking full use of the basic knowledge that was learned previously.We evaluate the proposed algorithm on two challenging MARL tasks:cooperative boxpushing and non-monotonic predator-prey.The experiment results have demonstrated the improved performance compared with state-of-the-art MARL algorithms.

Improving Operational Flexibility of Integrated Energy Systems Through Operating Conditions Optimization of CHP Units

As a type of energy system with bright application prospects,the integrated energy system(IES)is environmentally friendly and can improve overall energy efficiency.Tight coupling between heat and electricity outputs of combined heat and power(CHP)units limits IES operational flexibility significantly.To resolve this problem,in this paper,we integrate operating mode optimization of the natural gas combined cycle CHP unit(NGCC-CHP)into dispatch of the IES to improve flexibility of the IES.First,we analyze operational modes of the CHP units from the perspectives of thermal processes and physical mechanisms,including the adjustable extraction mode,backpressure mode,and switching mode.Next,we propose an explicit mathematical model for full-mode operation of the CHP units,in which the heat-electricity feasible region,switching constraints,and switching costs are all formulated in detail.Finally,a novel economic dispatch model is proposed for a heat and electricity IES,which uses the full-mode operation of CHP units to improve operational flexibility.The Fortuny-Amat transformation is used to convert the economic dispatch model into a mixed-integer quadratic programming model,which can then be solved using commercial solvers.Case studies demonstrate the proposed method can reduce operational costs and obviously promotes wind power utilization.

Machine Knowledge and Human Cognition

Intelligent machines are knowledge systems with unique knowledge structure and function.In this paper,we discuss issues including the characteristics and forms of machine knowledge,the relationship between knowledge and human cognition,and the approach to acquire machine knowledge.These issues are of great significance to the development of artificial intelligence.

Fast Carrier Selection of JPEG Steganography Appropriate for Application

In recent years,the improvement of the security of steganography mainly involves not only carrier security but also distortion function.In the actual environment,the existing method of carrier selection is limited by its complex algorithm and slow running speed,making it not appropriate for rapid communication.This study proposes a method for selecting carriers and improving the security of steganography.JPEG images are decompressed to spatial domain.Then correlation coefficients between two adjacent pixels in the horizontal,vertical,counter diagonal,and major diagonal directions are calculated.The mean value of the four correlation coefficients is used to evaluate the security of each JPEG image.The images with low correlation coefficients are considered safe carriers and used for embedding in our scheme.The experimental results indicate that the stego images generated from the selected carriers exhibit a higher anti-steganalysis capability than those generated from the randomly selected carriers.Under the premise of the same security level,the images with low correlation coefficients have a high capacity.Our algorithm has a very fast running speed,and the running time of a 2048×2048 image is less than 1 s.

Context-Aware Social Media User Sentiment Analysis

The user-generated social media messages usually contain considerable multimodal content.Such messages are usually short and lack explicit sentiment words.However,we can understand the sentiment associated with such messages by analyzing the context,which is essential to improve the sentiment analysis performance.Unfortunately,majority of the existing studies consider the impact of contextual information based on a single data model.In this study,we propose a novel model for performing context-aware user sentiment analysis.This model involves the semantic correlation of different modalities and the effects of tweet context information.Based on our experimental results obtained using the Twitter dataset,our approach is observed to outperform the other existing methods in analysing user sentiment.

ePUF: A Lightweight Double Identity Verification in IoT

Remote authentication is a safe and verifiable mechanism.In the Internet of Things (loT),remote hosts need to verify the legitimacy of identity of terminal devices.However,embedded devices can hardly afford sufficient resources for the necessary trusted hardware components.Software authentication with no hardware guarantee is generally vulnerable to various network attacks.In this paper,we propose a lightweight remote verification protocol.The protocol utilizes the unique response returned by Physical Unclonable Function (PUF) as legitimate identity basis of the terminal devices and uses quadratic residues to encrypt the PUF authentication process to perform a double identity verification scheme.Our scheme is secure against middleman attacks on the attestation response by preventing conspiracy attacks from forgery authentication.

IDEA:A Utility-Enhanced Approach to Incomplete Data Stream Anonymization

The prevalence of missing values in the data streams collected in real environments makes them impossible to ignore in the privacy preservation of data streams.However,the development of most privacy preservation methods does not consider missing values.A few researches allow them to participate in data anonymization but introduce extra considerable information loss.To balance the utility and privacy preservation of incomplete data streams,we present a utility-enhanced approach for Incomplete Data strEam Anonymization(IDEA).In this approach,a slide-window-based processing framework is introduced to anonymize data streams continuously,in which each tuple can be output with clustering or anonymized clusters.We consider the dimensions of attribute and tuple as the similarity measurement,which enables the clustering between incomplete records and complete records and generates the cluster with minimal information loss.To avoid the missing value pollution,we propose a generalization method that is based on maybe match for generalizing incomplete data.The experiments conducted on real datasets show that the proposed approach can efficiently anonymize incomplete data streams while effectively preserving utility.