In this study,a novel reinforcement learning task supervisor(RLTS)with memory in a behavioral control framework is proposed for human–multi-robot coordination systems(HMRCSs).Existing HMRCSs suffer from high decision...In this study,a novel reinforcement learning task supervisor(RLTS)with memory in a behavioral control framework is proposed for human–multi-robot coordination systems(HMRCSs).Existing HMRCSs suffer from high decision-making time cost and large task tracking errors caused by repeated human intervention,which restricts the autonomy of multi-robot systems(MRSs).Moreover,existing task supervisors in the null-space-based behavioral control(NSBC)framework need to formulate many priority-switching rules manually,which makes it difficult to realize an optimal behavioral priority adjustment strategy in the case of multiple robots and multiple tasks.The proposed RLTS with memory provides a detailed integration of the deep Q-network(DQN)and long short-term memory(LSTM)knowledge base within the NSBC framework,to achieve an optimal behavioral priority adjustment strategy in the presence of task conflict and to reduce the frequency of human intervention.Specifically,the proposed RLTS with memory begins by memorizing human intervention history when the robot systems are not confident in emergencies,and then reloads the history information when encountering the same situation that has been tackled by humans previously.Simulation results demonstrate the effectiveness of the proposed RLTS.Finally,an experiment using a group of mobile robots subject to external noise and disturbances validates the effectiveness of the proposed RLTS with memory in uncertain real-world environments.展开更多
基于区块链技术,提出具有身份认证和任务监管的声誉管理系统(Reputation Management System with Identity Authentication and Task Supervisor,RMS-IATS),解决群机器人内拜占庭机器人的识别问题,避免拜占庭机器人对群机器人造成安全威...基于区块链技术,提出具有身份认证和任务监管的声誉管理系统(Reputation Management System with Identity Authentication and Task Supervisor,RMS-IATS),解决群机器人内拜占庭机器人的识别问题,避免拜占庭机器人对群机器人造成安全威胁.首先,改进经典的基于区块链的群机器人声誉管理系统(Reputation Management System,RMS),引入惩罚因子,针对长期存在拜占庭行为的机器人实施更严厉的声誉值惩罚.其次,为了加快拜占庭机器人的识别速度,设计一种身份认证协议,将身份非法的机器人赋予一个较低的初始声誉值.再者,设计一种双层通信网络,用于机器人间的通信,解决群机器人系统因采用区块链技术带来的通信延迟问题.最后,通过仿真实验验证基于区块链的RMS-IATS和双层通信网络的有效性.相比经典的群机器人RMS,RMS-IATS在仿真模拟中识别不同类型拜占庭机器人所需的时间更短.相比使用区块链技术,在系统中使用双层通信网络进行通信时,可大幅减少系统的最大通信延迟.展开更多
基金supported by the National Natural Science Foundation of China(No.61603094)。
文摘In this study,a novel reinforcement learning task supervisor(RLTS)with memory in a behavioral control framework is proposed for human–multi-robot coordination systems(HMRCSs).Existing HMRCSs suffer from high decision-making time cost and large task tracking errors caused by repeated human intervention,which restricts the autonomy of multi-robot systems(MRSs).Moreover,existing task supervisors in the null-space-based behavioral control(NSBC)framework need to formulate many priority-switching rules manually,which makes it difficult to realize an optimal behavioral priority adjustment strategy in the case of multiple robots and multiple tasks.The proposed RLTS with memory provides a detailed integration of the deep Q-network(DQN)and long short-term memory(LSTM)knowledge base within the NSBC framework,to achieve an optimal behavioral priority adjustment strategy in the presence of task conflict and to reduce the frequency of human intervention.Specifically,the proposed RLTS with memory begins by memorizing human intervention history when the robot systems are not confident in emergencies,and then reloads the history information when encountering the same situation that has been tackled by humans previously.Simulation results demonstrate the effectiveness of the proposed RLTS.Finally,an experiment using a group of mobile robots subject to external noise and disturbances validates the effectiveness of the proposed RLTS with memory in uncertain real-world environments.
文摘基于区块链技术,提出具有身份认证和任务监管的声誉管理系统(Reputation Management System with Identity Authentication and Task Supervisor,RMS-IATS),解决群机器人内拜占庭机器人的识别问题,避免拜占庭机器人对群机器人造成安全威胁.首先,改进经典的基于区块链的群机器人声誉管理系统(Reputation Management System,RMS),引入惩罚因子,针对长期存在拜占庭行为的机器人实施更严厉的声誉值惩罚.其次,为了加快拜占庭机器人的识别速度,设计一种身份认证协议,将身份非法的机器人赋予一个较低的初始声誉值.再者,设计一种双层通信网络,用于机器人间的通信,解决群机器人系统因采用区块链技术带来的通信延迟问题.最后,通过仿真实验验证基于区块链的RMS-IATS和双层通信网络的有效性.相比经典的群机器人RMS,RMS-IATS在仿真模拟中识别不同类型拜占庭机器人所需的时间更短.相比使用区块链技术,在系统中使用双层通信网络进行通信时,可大幅减少系统的最大通信延迟.
