多智能体深度强化学习研究进展

深度强化学习(Deep Reinforcement Learning,DRL)在近年受到广泛的关注,并在各种领域取得显著的成功.由于现实环境通常包括多个与环境交互的智能体,多智能体深度强化学习(Multi-Agent Deep Reinforcement Learning,MADRL)获得蓬勃的发展,在各种复杂的序列决策任务上取得优异的表现.本文对多智能体深度强化学习的工作进展进行综述,主要内容分为三个部分.首先,我们回顾了几种常见的多智能体强化学习问题表示及其对应的合作、竞争和混合任务.其次,我们对目前的MADRL方法进行了全新的多维度的分类,并对不同类别的方法展开进一步介绍.其中,我们重点综述值函数分解方法,基于通信的MADRL方法以及基于图神经网络的MADRL方法.最后,我们研究了MADRL方法在现实场景中的主要应用.希望本文能够为即将进入这一快速发展领域的新研究人员和希望获得全方位了解并根据最新进展确定新方向的现有领域专家提供帮助.

A Heterogeneous Information Fusion Deep Reinforcement Learning for Intelligent Frequency Selection of HF Communication

The high-frequency(HF) communication is one of essential communication methods for military and emergency application. However, the selection of communication frequency channel is always a difficult problem as the crowded spectrum, the time-varying channels, and the malicious intelligent jamming. The existing frequency hopping, automatic link establishment and some new anti-jamming technologies can not completely solve the above problems. In this article, we adopt deep reinforcement learning to solve this intractable challenge. First, the combination of the spectrum state and the channel gain state is defined as the complex environmental state, and the Markov characteristic of defined state is analyzed and proved. Then, considering that the spectrum state and channel gain state are heterogeneous information, a new deep Q network(DQN) framework is designed, which contains multiple sub-networks to process different kinds of information. Finally, aiming to improve the learning speed and efficiency, the optimization targets of corresponding sub-networks are reasonably designed, and a heterogeneous information fusion deep reinforcement learning(HIF-DRL) algorithm is designed for the specific frequency selection. Simulation results show that the proposed algorithm performs well in channel prediction, jamming avoidance and frequency channel selection.

Handling Label Noise in Air Traffic Complexity Evaluation Based on Confident Learning and XGBoost

Air traffic complexity is a critical indicator for air traffic operation,and plays an important role in air traffic management(ATM),such as airspace reconfiguration,air traffic flow management and allocation of air traffic controllers(ATCos).Recently,many machine learning techniques have been used to evaluate air traffic complexity by constructing a mapping from complexity related factors to air traffic complexity labels.However,the low quality of complexity labels,which is named as label noise,has often been neglected and caused unsatisfactory performance in air traffic complexity evaluation.This paper aims at label noise in air traffic complexity samples,and proposes a confident learning and XGBoost-based approach to evaluate air traffic complexity under label noise.The confident learning process is applied to filter out noisy samples with various label probability distributions,and XGBoost is used to train a robust and high-performance air traffic complexity evaluation model on the different label noise filtered ratio datasets.Experiments are carried out on a real dataset from the Guangzhou airspace sector in China,and the results prove that the appropriate label noise removal strategy and XGBoost algorithm can effectively mitigate the label noise problem and achieve better performance in air traffic complexity evaluation.

Towards a Collaborative Learning Environment Through ICT： A Case Study

This paper expounds how the possibility of collaboration and construction of knowledge being put into practice in a group of ICT （information and communication technologies）-based teaching and learning programmes for Mother Tongue languages, collectively known as 10＇CMT. 10＇CMT, which is initiated by the ETD （Educational Technology Division） of MOE （Ministry of Education） Singapore, embodies a focus on the development of relevant pedagogy by which web-based technologies are embedded in meaningful learning activities in the classroom. Through a case study of a primary school in Singapore, this paper exemplifies how 10＇CMT has the ability to promote collective knowledge and, by doing so, essentially supporting the growth of the individual student＇s knowledge. It draws on the students＇ engagement in peer editing, peer evaluation, peer interaction, and feedback with self-reflective practices through the affordances of an array of online tools. This paper will also discuss how the 10＇CMT approach promotes the ability to respond flexibly to complex problems, to communicate effectively, to manage information, to work in teams, to use technology, and to produce new knowledge which are deemed to be crucial competencies for 21 st century.

Multicast Routing in Satellite Network

There are some problems in the dual-layer satellite MPLs metworks to be composed of LEO and MEO. In order to solve the problems, this paper presents a plan by means of unicast LSP to implement multicast in the dual-layer satellite MPLs networks. It has advantages of saving space and reducing extra charge.

Online support vector regression for reinforcement learning

The goal in reinforcement learning is to learn the value of state-action pair in order to maximize the total reward. For continuous states and actions in the real world, the representation of value functions is critical. Furthermore, the samples in value functions are sequentially obtained. Therefore, an online sup-port vector regression （OSVR） is set up, which is a function approximator to estimate value functions in reinforcement learning. OSVR updates the regression function by analyzing the possible variation of sup-port vector sets after new samples are inserted to the training set. To evaluate the OSVR learning ability, it is applied to the mountain-car task. The simulation results indicate that the OSVR has a preferable con- vergence speed and can solve continuous problems that are infeasible using lookup table.

Organizational Learning （OL） and Organizational Innovation （OI）： The Case of Information and Communication Technology （ICT） Industry in Malaysia

The progression through which a person acquires ,;kills, understanding, and opinions regarding a particular organization or company is called organizational learning （OL）. In this study, the connection between organizational innovation （OI） and OL within the inf3rmation and communication technology （ICT） industry in Malaysia is surveyed. These relationships are examined, because various previous inquiries have shown that an imperative precursor to firm performance is OL. Two hundred and seventy-eight surveys were completed by small and medium organizations across Malaysia. The connections existing between the causes of OL and the causes of OI were ascertained by using structural equation modeling （SEM）. Amongst the Malaysian small- and medium-sized enterprises （SMEs） that participated in fire study, OI and OL are considerably linked.

The Teaching Reform in Oral English for Non-English Majors Based on FIF

Nowadays the oral English levels of non-English majors in colleges do not live up to the expectations from the all walks of life. Due to the rapid development of the communication technology, many cell phone apps about the English learning have appeared. So this paper tries to put forward some effective ways to reform the oral English teaching for non-English majors based on FIF--a cell phone app about oral English learning after analyzing the status quo of current oral English teaching and learning and the advantages of FIF.

A Comparison of Different Communication Tools for Distance Learning in Nuclear Education

Recent advancement in nuclear education learning has been through the use of computers and simulation related tasks such as the use of industry codes. Further enhancements in nuclear education are being considered through the use of distance learning technologies. The purpose of this work is to explore distance learning related tools to determine if they can provide an enhanced learning environment for nuclear education. In this work, a set of tools are examined that can be used to augment or replace the traditional lecture method. These tools are Mediasite, Adobe Connect, Elluminate, and Camtasia. All four tools have recording capabilities that allow the students to experience the exchange of information in different ways. This paper compares recent experiences with each of these tools in providing nuclear engineering education and assesses the various constraints and impacts on delivery through direct feedback from students and instructors. In general, the tools were found to be useful for mature students on the condition that the lecturer was comfortable with the tools and in some cases, adequate support from IT groups was provided.

E-Learning Islamic Studies for Form Four Students

Despite the efforts by Ministry of Education to promote Information and Communication Technology （ICT） in education in Malaysia, the Islamic education syllabus is far behind the intended plan in ICT usage in learning and teaching. Concern was raised that Islamic Studies faced the risk of being misunderstood if the lessons were taught through self-accessing method with minimal intervention from teachers. Using the Dick and Carey instructional model as a framework, an e-learning version was devised for the national Form 4 Islamic Studies syllabus, ＂The steps and procedures of Hajj and Umrah＂. The Islamic Studies textbook for national secondary schools in Malaysia was reviewed using a systematic approach, from identifying the instructional goal through to formative and summative evaluation processes. Interview sessions with students were conducted to assess the developed e-learning Islamic Studies content. A subsequent survey with students was conducted. Results from the study indicated the e-learning Islamic Studies content had the potential to help students, being easy to use, and attracting and retaining students＇ attention.

ICT Paradox： Cost Efficiency of Web Based Learning

Indonesia government in this term Directorate General of Higher Education providing grants for ICT infrastructttres supplied through all bequest competitions. The kinds and amount of the grants are various which can be used to provide hardware to make ICT based teaching materials. The government issued a huge amount of funds, therefore, it must be balanced with an optimal utilization. This research aims to analyze cost efficiency of dual mode web-based learning in Indonesia. The object of this research is four higher educations in Indonesia. The analysis is done by comparing the average cost per student and the average cost per subject among 4 institutions. The result showed that there are institutions that issued higher cost compared with the others in producing the same learning media.

Simulations of Using Vectors in Natural Sciences Education

The implementation of ICT （information and communication technologies） into the educational process is becoming a reality in the 21st century. Today＇s students grow up with technology. To keep their attention, scientific problems should be solved through visualization, which is made possible using ICT in the educational process. In the modem educational process, students still have difficulties in learning science concepts. Also, it is a very common problem that students cannot apply mathematical language and concepts into other science areas such as physics, engineering, etc. For example, students start learning about vectors in mathematics in secondary school. Vectors are very important because they have a wide area of applications especially in physics, engineering and navigation to represent forces, tension, velocity, etc.. Using the free mathematical software GeoGebra, a simulation of using vectors in these areas is made. It will be shown that such simulations increase students＇ interest, keep their attention, and make this knowledge more real and more understandable and connected to the physical world and thus more applicable to their other studies.

The Video-Making Project： Why Students Love It？

It has been proven that ICT （information and communication technologies） affect the teaching and learning of a language, be it positive or negative. One of the examples of ICT that is used to enhance learners＇ creativity is the use of video. Although the use of ICT is encouraged in the learning and teaching of a language, it is still minimally applied in UMS （University Malaysia Sabah）, especially in the teaching and learning of a language. In this study, video-making is used as one of the assessments to measure learners＇ proficiency in the target language, which in this case, the Spanish Language. The video-making project was found to be able to improve learners＇ communication skills and knowledge building capabilities, as well as enhances learners＇ interest in the learning of the Spanish Language. It is found that intrinsic and extrinsic motivations, nature of the course, lecturers＇ positive support as well as students＇ positive determination to obtain good grades are among the factors contributing to the students＇ positive performance in the Spanish Language.

Study on the Reduced Traffic Congestion Method Based on Dynamic Guidance Information

This paper studies how to generate the reasonable information of travelers' decision in real network. This problem is very complex because the travelers' decision is constrained by different human behavior. The network conditions can be predicted by using the advanced dynamic OD(Origin-Destination, OD) estimation techniques. Based on the improved mesoscopic traffic model, the predictable dynamic traffic guidance information can be obtained accurately.A consistency algorithm is designed to investigate the travelers' decision by simulating the dynamic response to guidance information. The simulation results show that the proposed method can provide the best guidance information. Further,a case study is conducted to verify the theoretical results and to draw managerial insights into the potential of dynamic guidance strategy in improving traffic performance.