Targeted multi-agent communication algorithm based on state control

As an important mechanism in multi-agent interaction,communication can make agents form complex team relationships rather than constitute a simple set of multiple independent agents.However,the existing communication schemes can bring much timing redundancy and irrelevant messages,which seriously affects their practical application.To solve this problem,this paper proposes a targeted multiagent communication algorithm based on state control(SCTC).The SCTC uses a gating mechanism based on state control to reduce the timing redundancy of communication between agents and determines the interaction relationship between agents and the importance weight of a communication message through a series connection of hard-and self-attention mechanisms,realizing targeted communication message processing.In addition,by minimizing the difference between the fusion message generated from a real communication message of each agent and a fusion message generated from the buffered message,the correctness of the final action choice of the agent is ensured.Our evaluation using a challenging set of Star Craft II benchmarks indicates that the SCTC can significantly improve the learning performance and reduce the communication overhead between agents,thus ensuring better cooperation between agents.

A CSMA/CA based MAC protocol for hybrid Power-line/Visible-light communication networks:Design and analysis

Hybrid Power-line/Visible-light Communication(HPVC)network has been one of the most promising Cooperative Communication(CC)technologies for constructing Smart Home due to its superior communication reliability and hardware efficiency.Current research on HPVC networks focuses on the performance analysis and optimization of the Physical(PHY)layer,where the Power Line Communication(PLC)component only serves as the backbone to provide power to light Emitting Diode(LED)devices.So designing a Media Access Control(MAC)protocol remains a great challenge because it allows both PLC and Visible Light Communication(VLC)components to operate data transmission,i.e.,to achieve a true HPVC network CC.To solve this problem,we propose a new HPC network MAC protocol(HPVC MAC)based on Carrier Sense Multiple Access/Collision Avoidance(CSMA/CA)by combining IEEE 802.15.7 and IEEE 1901 standards.Firstly,we add an Additional Assistance(AA)layer to provide the channel selection strategies for sensor stations,so that they can complete data transmission on the selected channel via the specified CSMA/CA mechanism,respectively.Based on this,we give a detailed working principle of the HPVC MAC,followed by the construction of a joint analytical model for mathematicalmathematical validation of the HPVC MAC.In the modeling process,the impacts of PHY layer settings(including channel fading types and additive noise feature),CSMA/CA mechanisms of 802.15.7 and 1901,and practical configurations(such as traffic rate,transit buffer size)are comprehensively taken into consideration.Moreover,we prove the proposed analytical model has the solvability.Finally,through extensive simulations,we characterize the HPVC MAC performance under different system parameters and verify the correctness of the corresponding analytical model with an average error rate of 4.62%between the simulation and analytical results.

Resource Allocation and Power Control Policy for Device-to-Device Communication Using Multi-Agent Reinforcement Learning

Device-to-Device(D2D)communication is a promising technology that can reduce the burden on cellular networks while increasing network capacity.In this paper,we focus on the channel resource allocation and power control to improve the system resource utilization and network throughput.Firstly,we treat each D2D pair as an independent agent.Each agent makes decisions based on the local channel states information observed by itself.The multi-agent Reinforcement Learning(RL)algorithm is proposed for our multi-user system.We assume that the D2D pair do not possess any information on the availability and quality of the resource block to be selected,so the problem is modeled as a stochastic non-cooperative game.Hence,each agent becomes a player and they make decisions together to achieve global optimization.Thereby,the multi-agent Q-learning algorithm based on game theory is established.Secondly,in order to accelerate the convergence rate of multi-agent Q-learning,we consider a power allocation strategy based on Fuzzy C-means(FCM)algorithm.The strategy firstly groups the D2D users by FCM,and treats each group as an agent,and then performs multi-agent Q-learning algorithm to determine the power for each group of D2D users.The simulation results show that the Q-learning algorithm based on multi-agent can improve the throughput of the system.In particular,FCM can greatly speed up the convergence of the multi-agent Q-learning algorithm while improving system throughput.

Multi-User MmWave Beam Tracking via Multi-Agent Deep Q-Learning

Beamforming is significant for millimeter wave multi-user massive multi-input multi-output systems.In the meanwhile,the overhead cost of channel state information and beam training is considerable,especially in dynamic environments.To reduce the overhead cost,we propose a multi-user beam tracking algorithm using a distributed deep Q-learning method.With online learning of users’moving trajectories,the proposed algorithm learns to scan a beam subspace to maximize the average effective sum rate.Considering practical implementation,we model the continuous beam tracking problem as a non-Markov decision process and thus develop a simplified training scheme of deep Q-learning to reduce the training complexity.Furthermore,we propose a scalable state-action-reward design for scenarios with different users and antenna numbers.Simulation results verify the effectiveness of the designed method.

Neural-network-based fully distributed formation control for nonlinear multi-agent systems with event-triggered communication

This paper investigates the consensus-based formation control problem for multi-agent systems with unknown nonlinear dynamics.To achieve the desired formation,we propose two formation controllers to achieve the desired formation,one based on system states and the other on system outputs.The proposed controllers utilize adaptive gains to avoid global information and neural networks to estimate and compensate for nonlinearities.The proposed event-triggered schemes avoid continuous communication among agents and exclude the Zeno behavior.Stability analysis reveals that formation errors are bounded,and numerical simulations are used to validate the effectiveness of the proposed approaches.

Minimum-energy leader-following formation of distributed multiagent systems with communication constraints

This paper concerns minimum-energy leader-following formation design and analysis problems of distributed multiagent systems(DMASs)subjected to randomly switching topologies and aperiodic communication pauses.The critical feature of this paper is that the energy consumption during the formation control process is restricted by the minimum-energy constraint in the sense of the linear matrix inequality.Firstly,the leader-following formation control protocol is proposed based on the relative state information of neighboring agents,where the total energy consumption is considered.Then,minimum-energy leader-following formation design and analysis criteria are presented in the form of the linear matrix inequality,which can be checked by the generalized eigenvalue method.Especially,the value of the minimum-energy constraint is determined.An illustrative simulation is provided to show the effectiveness of the main results.

Collision Classification MAC Protocol for Underwater Acoustic Communication Networks Using Directional Antennas

Traditional underwater acoustic communication networks(UACNs)generally use omnidirectional transmission technology that causes a large number of data-packet collisions,thus resulting in low network throughput and high end-to-end delays.Compared with omnidirectional transmission technology,directional technology only sends and receives data packets in a specified direction.This can significantly reduce the probability of collisions and improve network performance.However,it also causes a deafness problem,which occurs when the sending node sends a data packet to the receiving node but the receiving node is unable to reply to the sender,because its antenna beam is closed.To resolve this issue,this study proposes a collision classification media access control(CC-MAC)protocol for UACNs.With this protocol,the underwater acoustic channel is divided into two subchannels,and the nodes transmit corresponding data types on them.The sending node can estimate the current status of the receiving node(i.e.,no collision,normal collision,deafness)according to the type of the data packet received and the sub-channel it arrived on,and it can choose correct options to improve network efficiency.Finally,we verify the performance of CC-MAC via simulations,showing that the protocol achieved higher network throughput and lower end-toend delays.

Necessary and Sufficient Conditions for Consensus in Third Order Multi-Agent Systems

We deal with a consensus control problem for a group of third order agents which are networked by digraphs.Assuming that the control input of each agent is constructed based on weighted difference between its states and those of its neighbor agents, we aim to propose an algorithm on computing the weighting coefficients in the control input. The problem is reduced to designing Hurwitz polynomials with real or complex coefficients. We show that by using Hurwitz polynomials with complex coefficients, a necessary and sufficient condition can be obtained for designing the consensus algorithm. Since the condition is both necessary and sufficient, we provide a kind of parametrization for all the weighting coefficients achieving consensus. Moreover, the condition is a natural extension to second order consensus, and is reasonable and practical due to its comparatively decreased computation burden. The result is also extended to the case where communication delay exists in the control input.

Event-Triggered Differentially Private Average Consensus for Multi-agent Network

This paper investigates the differentially private problem of the average consensus for a class of discrete-time multi-agent network systems(MANSs). Based on the MANSs,a new distributed differentially private consensus algorithm(DPCA) is developed. To avoid continuous communication between neighboring agents, a kind of intermittent communication strategy depending on an event-triggered function is established in our DPCA. Based on our algorithm, we carry out the detailed analysis including its convergence, its accuracy, its privacy and the trade-off between the accuracy and the privacy level, respectively. It is found that our algorithm preserves the privacy of initial states of all agents in the whole process of consensus computation. The trade-off motivates us to find the best achievable accuracy of our algorithm under the free parameters and the fixed privacy level. Finally, numerical experiment results testify the validity of our theoretical analysis.

Autonomous Optimal Coordination Scheme in Protection System of Power Distribution Network by Using Multi-Agent Concept

A protection system using a multi-agent concept for power distribution networks is proposed.Every digital over current relay(OCR)is developed as an agent by adding its own intelligence,self-tuning and communication ability.The main advantage of the multi-agent concept is that a group of agents work together to achieve a global goal which is beyond the ability of each individual agent.In order to cope with frequent changes in the network operation condition and faults,an OCR agent,proposed in this paper,is able to detect a fault or a change in the network and find its optimal parameters for protection in an autonomous manner considering information of the whole network obtained by communication between other agents.Through this kind of coordination and information exchanges,not only a local but also a global protective scheme is completed.Simulations in a simple distribution network show the effectiveness of the proposed protection system.

A Novel Medium Access Control Protocol for Real-Time Wireless Communications in Industrial Automation

In this paper, a novel Medium Access Control (MAC) protocol for industrial Wireless Local Area Networks (WLANs) is proposed and studied. The main challenge in industry automation systems is the ultra-low network latency with a target upper bound in the order of 1 ms while maintaining high network reliability and availability. The novelty of the proposed wireless MAC protocol resides in its similar latency performance as its counterpart in wired industrial LAN. First, the functional design of the MAC protocol is introduced. Then its performance results gained from hardware implementation (SystemC and VHDL) on an FPGA platform are presented. Finally, a real-time communication module which achieves the ultra-low latency required in industrial automation is described.

Containment consensus with measurement noises and time-varying communication delays

In this paper,we consider the containment consensus control problem for multi-agent systems with measurement noises and time-varying communication delays under directed networks.By using stochastic analysis tools and algebraic graph theory,we prove that the followers can converge to the convex hull spanned by the leaders in the sense of mean square if the allowed upper bound of the time-varying delays satisfies a certain sufficient condition.Moreover,the time-varying delays are asymmetric for each follower agent,and the time-delay-dependent consensus condition is derived.Finally,numerical simulations are provided to illustrate the effectiveness of the obtained theoretical results.

Mean-square composite-rotating consensus of second-order systems with communication noises

We study the mean-square composite-rotating consensus problem of second-order multi-agent systems with communication noises, where all agents rotate around a common center and the center of rotation spins around a fixed point simultaneously. Firstly, a time-varying consensus gain is introduced to attenuate to the effect of communication noises. Secondly, sufficient conditions are obtained for achieving the mean-square composite-rotating consensus. Finally, simulations are provided to demonstrate the effectiveness of the proposed algorithm.

Very Low-power Wireless Communication Technology-ZigBee

The ZigBee as the main supporting technology for the wireless sensor network has received extensive attention. Completed ZigBee protocol kits consist of the high-level application specification, the application convergence layer, the network layer, the data link layer, and the Physical (PHY) layer. The protocols of the network layer and higher ones are set up by the ZigBee Alliance, the physical layer and the Media Access Control (MAC) layer are defined by the IEEE 802.15.4 standard. The IEEE 802.15.4 physical layer simply adopts technologies of mapping from bit to symbol, conversion from symbol to chip sequence, and Offset Quadrature Phase Shift Keying (OQPSK) modulation, avoiding complicated arithmetic such as channel coding. The IEEE 802.15.4 MAC layer adopts Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) technology and supports the sleeping mode. The ZigBee enables low data transmission rate, low power consumption at low cost, and is more suitable for industry monitoring systems, sensor networks, home monitoring systems, and security systems.

Optimal Secondary Control of Islanded AC Microgrids with Communication Time-delay Based on Multi-agent Deep Reinforcement Learning

In this paper,an optimal secondary control strategy is proposed for islanded AC microgrids considering communi-cation time-delays.The proposed method is designed based on the data-driven principle,which consists of an offine training phase and online application phase.For offline training,each control agent is formulated by a deep neural network(DNN)and trained based on a multi-agent deep reinforcement learning(MA-DRL)framework.A deep deterministic policy gradient(DDPG)algorithm is improved and applied to search for an optimal policy of the secondary control,where a global cost function is developed to evaluate the overall control performance.In addition,the communication time-delay is introduced in the system to enrich training scenarios,which aims to solve the time-delay problem in the secondary control.For the online stage,each controller is deployed in a distributed way which only requires local and neighboring information for each DG.Based on this,the well-trained controllers can provide optimal solutions under load variations,and communication time-delays for online applications.Several case studies are conducted to validate the feasibility and stability of the proposed secondary control.Index Terms-Communication time-delay,global cost function,islanded AC microgrid,multi-agent deep reinforcement learning(MA-DRL),secondary control.

CM-MAC:一种基于分簇的多信道车载网MAC协议

车载网VANET是一种应用于智能交通系统的新型无线移动自组织网络(mobile ad hoc network,MANET).随着车辆以及移动ad hoc网络技术的发展,车载网已经成为一个新兴的研究领域.针对VANET中车辆行驶的特征,提出一种拓扑相对稳定的车辆分簇算法.在此算法基础上,根据专用短程通信(dedicated short range communication,DSRC)标准中控制信道(control channel,CCH)和服务信道(service channel,SCH)的分配,考虑车辆间的无线通信干扰和不同应用的QoS需求,提出一种基于分簇的多信道混合型MAC协议,簇内通信采用非竞争的TDMA机制,簇间通信采用基于竞争的CSMA?CA机制,相邻簇采用不同的服务信道.模拟实验表明,提出的MAC协议在同时满足实时应用的延迟需求和非实时应用的吞吐量方面,优于现有协议.

A fully distributed robust optimal control approach for air-conditioning systems considering uncertainties of communication link in IoT-enabled building automation systems

Internet of Things(IoT)technologies are increasingly implemented in buildings as the cost-effective smart sens-ing infrastructure of building automation systems(BASs).They are also dispersed computing resources for novel distributed optimal control approaches.However,wireless communication networks are critical to fulfill these tasks with the performance influenced by inherent uncertainties in networks,e.g.,unpredictable occurrence of link failures.Centralized and hierarchical distributed approaches are vulnerable against link failure,while the robustness of fully distributed approaches depends on the algorithms adopted.This study therefore proposes a fully distributed robust optimal control approach for air-conditioning systems considering uncertainties of com-munication link in IoT-enabled BASs.The distributed algorithm is adopted that agents know their out-neighbors only.Agents directly coordinate with the connected neighbors for global optimization.Tests are conducted to test and validate the proposed approach by comparing with existing approaches,i.e.,the centralized,the hierarchical distributed and the fully distributed approaches.Results show that different approaches are vulnerable against to uncertainties of communication link to different extents.The proposed approach always guarantees the optimal control performance under normal conditions and conditions with link failures,verifying its high robustness.It also has low computation complexity and high optimization efficiency,thus applicable on IoT-enabled BASs.

A Fairness-Enhanced Intelligent MAC Scheme Using Q-Learning-Based Bidirectional Backoff for Distributed Vehicular Communication Networks

With the increasing attention to front-edge vehicular communication applications,distributed resource allocation is beneficial to the direct communications between vehicle nodes.However,in highly dynamic distributed vehicular networks,quality of service(QoS)of the systems would degrade dramatically because of serious packet collisions in the absence of sufficient link knowledge.Focusing on the fairness optimization,a Q-learning-based collision avoidance(QCA)scheme,which is characterized by an ingenious bidirectional backoff reward model RQCA corresponding to arbitrary backoff stage transitions,has been proposed in an intelligent distributed media access control protocol.In QCA,an intelligent bidirectional backoff agent based on the Markov decision process model can actively motivate each vehicle agent to update itself toward an optimal backoff sub-intervel BSIopt through either positive or negative bidirectional transition individually,resulting in the distinct fair communication with a proper balance of the resource allocation.According to the reinforcement learning theory,the problem of goodness evaluation on the backoff stage self-selection policy is equal to the problem of maximizing Q function of the vehicle in the current environment.The final decision on BSI_(opt) related to an optimal contention window range was solved through maximizing the Q value or Q_(max).The ε-greedy algorithm was used to keep a reasonable convergence of the Q_(max) solution.For the fairness evaluation of QCA,four kinds of dynamic impacts on the vehicular networks were investigated:mobility,density,payload size,and data rate with a network simulator NS2.Consequently,QCA can achieve fair communication efficiently and robustly,with advantages of superior Jain’s fairness index,relatively high packet delivery ratio,and low time delay.

Consensus of cyber-physical power systems based on multi-agent systems with communication constraints

The consensus protocol of cyber-physical power systems is proposed based on fractional-order multi-agent systems with communication constraints.It aims to enable each generator to reach a time-varying common rotor angle and rotor speed.Communication constraints including event-triggered sampling and partial information transmission are considered to render the consensus protocol more realistic.The Zeno behavior is excluded during the system sampling process.A sufficient condition is derived to solve the consensus problem.The effectiveness of the proposed consensus protocol is demonstrated by a numerical example.

CONSENSUS DESIGN FOR CONTINUOUS-TIME MULTI-AGENT SYSTEMS WITH COMMUNICATION DELAY

This paper investigates a consensus design problem for continuous-time first-order multiagent systems with uniform constant communication delay.Provided that the agent dynamic is unstable and the diagraph is undirected,sufficient conditions are derived to guarantee consensus.The key technique is the adoption of historical input information in the protocol.Especially,when agent's own historical input information is used in the protocol design,the consensus condition is constructed in terms of agent dynamic,communication delay,and the eigenratio of the network topology.Simulation result is presented to validate the effectiveness of the theoretical result.