Role Dynamic Allocation of Human-Robot Cooperation Based on Reinforcement Learning in an Installation of Curtain Wall

A real-time adaptive roles allocation method based on reinforcement learning is proposed to improve humanrobot cooperation performance for a curtain wall installation task.This method breaks the traditional idea that the robot is regarded as the follower or only adjusts the leader and the follower in cooperation.In this paper,a self-learning method is proposed which can dynamically adapt and continuously adjust the initiative weight of the robot according to the change of the task.Firstly,the physical human-robot cooperation model,including the role factor is built.Then,a reinforcement learningmodel that can adjust the role factor in real time is established,and a reward and actionmodel is designed.The role factor can be adjusted continuously according to the comprehensive performance of the human-robot interaction force and the robot’s Jerk during the repeated installation.Finally,the roles adjustment rule established above continuously improves the comprehensive performance.Experiments of the dynamic roles allocation and the effect of the performance weighting coefficient on the result have been verified.The results show that the proposed method can realize the role adaptation and achieve the dual optimization goal of reducing the sum of the cooperator force and the robot’s Jerk.

Regional Multi-Agent Cooperative Reinforcement Learning for City-Level Traffic Grid Signal Control

This article studies the effective traffic signal control problem of multiple intersections in a city-level traffic system.A novel regional multi-agent cooperative reinforcement learning algorithm called RegionSTLight is proposed to improve the traffic efficiency.Firstly a regional multi-agent Q-learning framework is proposed,which can equivalently decompose the global Q value of the traffic system into the local values of several regions Based on the framework and the idea of human-machine cooperation,a dynamic zoning method is designed to divide the traffic network into several strong-coupled regions according to realtime traffic flow densities.In order to achieve better cooperation inside each region,a lightweight spatio-temporal fusion feature extraction network is designed.The experiments in synthetic real-world and city-level scenarios show that the proposed RegionS TLight converges more quickly,is more stable,and obtains better asymptotic performance compared to state-of-theart models.

The Effectiveness of Group Cooperative Learning Method in Badminton Teaching in Colleges and Universities

In college badminton teaching,teachers utilize the group cooperative learning method,which not only helps to improve students’badminton skill level but also cultivates their teamwork spirit,communication skills,and self-management ability unconsciously.In view of this,this paper mainly describes the significance of applying the group cooperative learning method in college badminton teaching,analyzes the current problems in college badminton teaching,and aims to discover effective development strategies for group cooperative learning method in college badminton teaching in order to improve the effectiveness of college badminton teaching.

Application of Cooperative Learning and Flipped Classrooms in University Basketball Courses in China

This study investigates the application of the teaching model combining cooperative learning and flipped classrooms in university basketball courses in China.By analyzing the advantages and disadvantages of the traditional basketball teaching model and students’satisfaction with the course,the necessity of implementing cooperative learning and flipped classrooms is proposed.The study planned in detail the implementation strategies before class,in the classroom,and after class,and compared them with the control group through an experimental design.The experimental results showed that the new teaching mode demonstrated significant advantages in terms of learning outcomes,student satisfaction,and teacher evaluation.This study provides a valuable reference for the future reform of the physical education curriculum.

A Whale Optimization Algorithm with Distributed Collaboration and Reverse Learning Ability

Due to the development of digital transformation,intelligent algorithms are getting more and more attention.The whale optimization algorithm(WOA)is one of swarm intelligence optimization algorithms and is widely used to solve practical engineering optimization problems.However,with the increased dimensions,higher requirements are put forward for algorithm performance.The double population whale optimization algorithm with distributed collaboration and reverse learning ability(DCRWOA)is proposed to solve the slow convergence speed and unstable search accuracy of the WOA algorithm in optimization problems.In the DCRWOA algorithm,the novel double population search strategy is constructed.Meanwhile,the reverse learning strategy is adopted in the population search process to help individuals quickly jump out of the non-ideal search area.Numerical experi-ments are carried out using standard test functions with different dimensions(10,50,100,200).The optimization case of shield construction parameters is also used to test the practical application performance of the proposed algo-rithm.The results show that the DCRWOA algorithm has higher optimization accuracy and stability,and the convergence speed is significantly improved.Therefore,the proposed DCRWOA algorithm provides a better method for solving practical optimization problems.

A review of mobile robot motion planning methods:from classical motion planning workflows to reinforcement learning-based architectures

Motion planning is critical to realize the autonomous operation of mobile robots.As the complexity and randomness of robot application scenarios increase,the planning capability of the classical hierarchical motion planners is challenged.With the development of machine learning,the deep reinforcement learning(DRL)-based motion planner has gradually become a research hotspot due to its several advantageous feature.The DRL-based motion planner is model-free and does not rely on the prior structured map.Most importantly,the DRL-based motion planner achieves the unification of the global planner and the local planner.In this paper,we provide a systematic review of various motion planning methods.Firstly,we summarize the representative and state-of-the-art works for each submodule of the classical motion planning architecture and analyze their performance features.Then,we concentrate on summarizing reinforcement learning(RL)-based motion planning approaches,including motion planners combined with RL improvements,map-free RL-based motion planners,and multi-robot cooperative planning methods.Finally,we analyze the urgent challenges faced by these mainstream RLbased motion planners in detail,review some state-of-the-art works for these issues,and propose suggestions for future research.

Promotion of Cooperation in a Spatial Public Goods Game with Long Range Learning and Mobility

We studied the effect of population density in a spatial public goods game.We found that the effect on the evolution of cooperation is very complex when the strategy learning and mobility of players in a long range are considered in a two-dimensional lattice.As the learning range is larger than the mobility range,the system is driven to enter into a cooperation state for a low population density,because a small local group is beneficial to sustain a high level of cooperation.As population density increases to a moderate range,the mobility of players from a domain invaded by defectors supports the evolution stability of cooperation.When the mobility range is larger than the learning range,a formation of compact domains of cooperators promotes cooperation as the population density becomes high.

Learning-Based Joint Service Caching and Load Balancing for MEC Blockchain Networks

Integrating the blockchain technology into mobile-edge computing(MEC)networks with multiple cooperative MEC servers(MECS)providing a promising solution to improving resource utilization,and helping establish a secure reward mechanism that can facilitate load balancing among MECS.In addition,intelligent management of service caching and load balancing can improve the network utility in MEC blockchain networks with multiple types of workloads.In this paper,we investigate a learningbased joint service caching and load balancing policy for optimizing the communication and computation resources allocation,so as to improve the resource utilization of MEC blockchain networks.We formulate the problem as a challenging long-term network revenue maximization Markov decision process(MDP)problem.To address the highly dynamic and high dimension of system states,we design a joint service caching and load balancing algorithm based on the double-dueling Deep Q network(DQN)approach.The simulation results validate the feasibility and superior performance of our proposed algorithm over several baseline schemes.

A dynamic incentive and reputation mechanism for energy-efficient federated learning in 6G

As 5G becomes commercial,researchers have turned attention toward the Sixth-Generation(6G)network with the vision of connecting intelligence in a green energy-efficient manner.Federated learning triggers an upsurge of green intelligent services such as resources orchestration of communication infrastructures while preserving privacy and increasing communication efficiency.However,designing effective incentives in federated learning is challenging due to the dynamic available clients and the correlation between clients'contributions during the learning process.In this paper,we propose a dynamic incentive and reputation mechanism to improve energy efficiency and training performance of federated learning.The proposed incentive based on the Stackelberg game can timely adjust optimal energy consumption with changes in available clients during federated learning.Meanwhile,clients’contributions in reputation management are formulated based on the cooperative game to capture the correlation between tasks,which satisfies availability,fairness,and additivity.The simulation results show that the proposed scheme can significantly motivate high-performance clients to participate in federated learning and improve the accuracy and energy efficiency of the federated learning model.

Cooperative multi-target hunting by unmanned surface vehicles based on multi-agent reinforcement learning

To solve the problem of multi-target hunting by an unmanned surface vehicle(USV)fleet,a hunting algorithm based on multi-agent reinforcement learning is proposed.Firstly,the hunting environment and kinematic model without boundary constraints are built,and the criteria for successful target capture are given.Then,the cooperative hunting problem of a USV fleet is modeled as a decentralized partially observable Markov decision process(Dec-POMDP),and a distributed partially observable multitarget hunting Proximal Policy Optimization(DPOMH-PPO)algorithm applicable to USVs is proposed.In addition,an observation model,a reward function and the action space applicable to multi-target hunting tasks are designed.To deal with the dynamic change of observational feature dimension input by partially observable systems,a feature embedding block is proposed.By combining the two feature compression methods of column-wise max pooling(CMP)and column-wise average-pooling(CAP),observational feature encoding is established.Finally,the centralized training and decentralized execution framework is adopted to complete the training of hunting strategy.Each USV in the fleet shares the same policy and perform actions independently.Simulation experiments have verified the effectiveness of the DPOMH-PPO algorithm in the test scenarios with different numbers of USVs.Moreover,the advantages of the proposed model are comprehensively analyzed from the aspects of algorithm performance,migration effect in task scenarios and self-organization capability after being damaged,the potential deployment and application of DPOMH-PPO in the real environment is verified.

双路径合作的原型矫正小样本分类模型

基于度量的元学习在学习过程中存在由于稀缺数据分布导致习得的先验知识不足、从样本中提取到的单一视图特征易受弱相关或无关特征的干扰以及因分类造成的代表性特征偏差的问题。针对这些问题,提出了一种双路径合作的原型矫正小样本分类模型。首先,通过双路径合作模块从多视图角度自适应地突出关键特征和弱化弱相关特征,充分利用特征信息获得先验知识来提升特征的表达能力;其次,通过基于查询集样本特征信息的原型矫正分类策略来解决类内原型的偏差问题;最后,通过损失函数反向更新模型参数,模型分类准确率得以提升。在五个公开的数据集上进行了5-way 1-shot和5-way 5-shot对比实验,较基准模型而言,在miniImageNet数据集上,准确率提升了5.57个百分点和3.90个百分点;在tieredImageNet数据集上,准确率提升了5.68个百分点和3.93个百分点;在CUB数据集上,准确率提升了6.93个百分点和3.13个百分点;在CIFAR-FS数据集上,准确率提升了8.03个百分点和1.65个百分点;在FC-100数据集上,准确率提升了4.25个百分点和4.89个百分点。实验结果表明,提出的双路径合作的原型矫正小样本分类模型能在小样本学习领域有良好的性能,且模型中的模块可迁移到其他模型中使用。

基于深度强化学习的SCR脱硝系统协同控制策略研究

针对选择性催化还原(SCR)脱硝系统大惯性、多扰动等特点,提出了一种基于多维状态信息和分段奖励函数优化的深度确定性策略梯度(DDPG)协同比例积分微分(PID)控制器的控制策略。针对SCR脱硝系统中存在部分可观测马尔可夫决策过程(POMDP),导致DDPG算法策略学习效率较低的问题,首先设计SCR脱硝系统的多维状态信息;其次,设计SCR脱硝系统的分段奖励函数;最后,设计DDPG-PID协同控制策略,以实现SCR脱硝系统的控制。结果表明:所设计的DDPG-PID协同控制策略提高了DDPG算法的策略学习效率,改善了PID的控制效果,同时具有较强的设定值跟踪能力、抗干扰能力和鲁棒性。

真学习还是假参与?——Presentation在高校课堂教学应用中的问题与改进

Presentation是目前国内高校教学改革中常用的教学形式之一,在各类学科教学中均有不同程度的应用。这种以增强教学双主体互动为主要特征的教学形式,在实际教学中呈现出学生课堂参与深度不足、学习收获不系统、课堂汇报“一言堂”等问题。学生学习是为了完成任务而非获得知识、学生习惯于服从教师权威而非批判性思考、教师教学准备不足与指导环节缺失,以及教学评价的评价主体与形式单一是造成Presenta-tion应用效果不佳的重要因素。提高Presentation的应用效果,需要强化对教学过程的管理,保障Presentation规范实施;改革学业评价方式,促进学生主动思考与合作学习;拓展教学资源供给,为学生自主与合作学习提供更多支持;鼓励教师开展教学行动研究,不断提高教学专业化水平。

结构抗震设计课程产学合作协同育人教学模式构建

教育部产学合作协同育人教学内容和课程体系改革项目,旨在将人才培养的最新要求引入教学过程,推动高校更新教学内容、完善课程体系,建设适应行业发展需要、可共享的资源并推广应用。顺应国家战略要求,针对目前土木工程专业理论课程教学中普遍存在的问题和结构抗震设计的课程特点,基于混合式教学方法构建产学合作协同育人教学模式,探讨教学内容的调整优化、典型结构分析软件的引入、课程大作业的设计、试验项目的选取、考核方式以及教研试验平台的建立和利用等教学改革措施,描述了该教学模式的主要思路和关键实施步骤,列举了在理论知识、工程应用、创新思想和专业素质方面的收获和启发,验证了产学合作协同育人教学模式的良好效果。

跨机构联邦学习的激励机制综述

联邦学习作为一种分布式机器学习,有效地解决了大数据时代的数据共享难题。其中,跨机构联邦学习是机构之间互相合作的一种联邦学习类型。如何在跨机构合作的过程中设计合理的激励机制十分重要。文中从跨机构合作的角度,对现有的跨机构联邦学习的激励机制研究进行了综述。首先介绍跨机构合作过程中的3个基本问题,即高隐私性、数据异质性、公平性,然后分析了以全局模型为中心和以参与者为中心这两种不同的跨机构合作模式下的激励机制设计方法,最后总结了影响跨机构合作稳定发展的几个影响因素,即参与者的数据演变、参与者合作关系变动和参与者的负面行为,并展望了跨机构联邦合作的未来方向。

基于深度学习的认知物联网频谱感知算法研究

针对认知物联网(Internet of Things, IoT)对低信噪比(Signal to Noise Ratio, SNR)的频谱感知性能低下以及传统卷积神经网络(Convolutional Neural Network, CNN)频谱感知方法提取数据特征不充分导致感知性能差等问题,提出了一种改进残差网络——ResNeXt的单节点频谱感知算法,ResNeXt只需要设置少量超参数且高度模块化,将该网络在图像处理上的优势应用在频谱感知问题上,先将接收信号转成二维矩阵并归一灰度化处理,得到灰度图像作为网络的输入。通过训练ResNeXt来提取灰度图像特征,将在线数据输入完成频谱感知。将各个次用户(Secondary User, SU)得到的评分向量矩阵直接用融合中心SoftCombinationNet(SCN)融合获得协作频谱感知结果,有效解决了传统硬融合方法检测性能低、软融合处理复杂等问题。实验结果表明,所提方法在低SNR仍能实现低虚警率、高检测概率,优于传统频谱感知方法。

基于实时互动的社交媒体在高校教学中的应用研究——以《企业管理概论》课程为例

随着移动互联网的快速发展,社交媒体已经成为人们日常生活中不可或缺的组成部分,社交媒体使用对高校学生学习效果的影响,在高等教育教学领域逐渐引起学者的关注。基于社交媒体应用与合作学习视角,讨论社交媒体在高校教学中的应用,结合高校学生使用社交媒体的学习动机、学习行为及学习效果,从教学设计、教学实施、教学评价及教学反思四个方面,提出在高校教学中合理应用社交媒体辅助课程教学,促进学生合作学习,提升学习效果及协作能力。

重内涵、强实践、塑思维、促合作的模电课程

“模拟电子技术”课程是信息类专业重要的学科基础平台课,具有复杂性、工程性、实践性等特点,被学生戏称为“魔鬼”电路;针对学生学习该课程存在的客观“屏障”和主观问题,剖析复杂根源——除“魔”、构建个性实践——贯“通”、创设自主阵地——深“思”、构筑合作生态——同“进”,由此积累探索出点、线、面、体多维度“模电”教学架构,从而实现重内涵、强实践、塑思维、促合作的教学目标,全面提升学生的高阶思维和创新实践能力。

基于案例分析和小组合作模式的翻转课堂在环境卫生学课程中的应用效果

目的探讨基于案例分析和小组合作模式的翻转课堂教学模式在环境卫生学课程教学中的应用效果。方法选取哈尔滨医科大学2015至2019级183名公共事业管理专业本科生作为研究对象,将2015至2017级99名学生作为对照组,2018至2019级84名学生作为观察组。对照组采用传统教学模式,观察组采用基于案例分析和小组合作模式的翻转课堂教学模式。比较两组学习成绩、成绩优良率,并分析观察组2019级学生对翻转课堂教学模式效果的评价。结果观察组学习成绩为(85.30±8.72)分,高于对照组的(77.82±9.31)分,差异有统计学意义(P<0.05)。观察组成绩优良率为(75.33±5.95)%,高于对照组的(33.98±7.02)%,差异有统计学意义(P<0.05)。调查问卷结果显示,45名2019级学生中,89.74%的学生愿意自主学习教师布置的任务单,74.36%的学生认为学习任务单难度适中,97.44%的学生认为学习任务单对课堂任务有帮助,89.74%的学生认为学习任务单时间满意度好,89.74%的学生认为翻转课堂教学有助于提高知识的掌握度,94.88%的学生认为翻转课堂有利于解决实际问题;92.31%的学生喜欢翻转课堂的教学模式,≥80.00%的学生认为翻转课堂可提高语言表达能力(84.62%)、自主学习能力(92.31%)、沟通能力(92.31%)和团结协作能力(94.87%),增强学习兴趣(84.62%)。结论基于案例和小组合作的翻转课堂教学模式有助于提升教学效果,在多方面对学生能力的发展具有积极影响,进而可促进预防医学教学改革。