Toward Trustworthy Decision-Making for Autonomous Vehicles:A Robust Reinforcement Learning Approach with Safety Guarantees

While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present a novel robust reinforcement learning approach with safety guarantees to attain trustworthy decision-making for autonomous vehicles.The proposed technique ensures decision trustworthiness in terms of policy robustness and collision safety.Specifically,an adversary model is learned online to simulate the worst-case uncertainty by approximating the optimal adversarial perturbations on the observed states and environmental dynamics.In addition,an adversarial robust actor-critic algorithm is developed to enable the agent to learn robust policies against perturbations in observations and dynamics.Moreover,we devise a safety mask to guarantee the collision safety of the autonomous driving agent during both the training and testing processes using an interpretable knowledge model known as the Responsibility-Sensitive Safety Model.Finally,the proposed approach is evaluated through both simulations and experiments.These results indicate that the autonomous driving agent can make trustworthy decisions and drastically reduce the number of collisions through robust safety policies.

A Combined Reinforcement Learning and Model Predictive Control for Car-Following Maneuver of Autonomous Vehicles

Model predictive control is widely used in the design of autonomous driving algorithms.However,its parameters are sensitive to dynamically varying driving conditions,making it difficult to be implemented into practice.As a result,this study presents a self-learning algorithm based on reinforcement learning to tune a model predictive controller.Specifically,the proposed algorithm is used to extract features of dynamic traffic scenes and adjust the weight coefficients of the model predictive controller.In this method,a risk threshold model is proposed to classify the risk level of the scenes based on the scene features,and aid in the design of the reinforcement learning reward function and ultimately improve the adaptability of the model predictive controller to real-world scenarios.The proposed algorithm is compared to a pure model predictive controller in car-following case.According to the results,the proposed method enables autonomous vehicles to adjust the priority of performance indices reasonably in different scenarios according to risk variations,showing a good scenario adaptability with safety guaranteed.

Autonomous machine learning for early bot detection in the internet of things

The high costs incurred due to attacks and the increasing number of different devices in the Internet of Things(IoT)highlight the necessity of the early detection of botnets(i.e.,a network of infected devices)to gain an advantage against attacks.However,early botnet detection is challenging because of continuous malware mutations,the adoption of sophisticated obfuscation techniques,and the massive volume of data.The literature addresses botnet detection by modeling the behavior of malware spread,the classification of malicious traffic,and the analysis of traffic anomalies.This article details ANTE,a system for ANTicipating botnEt signals based on machine learning algorithms.The system adapts itself to different scenarios and detects different types of botnets.It autonomously selects the most appropriate Machine Learning(ML)pipeline for each botnet and improves the classification before an attack effectively begins.The system evaluation follows trace-driven experiments and compares ANTE results to other relevant results from the literature over four representative datasets:ISOT HTTP Botnet,CTU-13,CICDDoS2019,and BoT-IoT.Results show an average detection accuracy of 99.06%and an average bot detection precision of 100%.

An Investigation of College English Autonomous Learning in Network Multimodal Context

In the current society, based on the growing development of network information technology, the teaching in many colleges and universities has also introduced it to adapt to the situation. This trend can provide more useful conditions for students to learn, which requires students to master enough self-learning abilities to adapt to this model. The study in the paper shows that students are usually interested in autonomous learning in a multimodal environment, but the degree of strategy choice is relatively low, and the learning process is blind and passive with the lack of self-confidence. Facing the future, schools should actively integrate into network thinking, and teachers should change their roles and train and guide students’ learning strategies and learning motivations, so as to achieve better teaching results.

Research on College Studentst'Autonomous EFL Learning Affer Course Exemption

By analyzing the English learning logs of 12 students in a provincial university in south-west China after they had been exempted from taking college English courses,this study investigated college students’autonomous EFL(English as a foreign language)learning after course exemption,including the use of mediational means in EFL learning,EFL learning hours,and other factors affecting EFL learning,in the hope of giving new perspectives on college ELF curriculum design,teaching,and education management.

Oral English Autonomous Learning Ability of English Majors Under the Background of the Internet and Its Improvement

The Internet is an important means of communication for contemporary college students,especially those majoring in English,to acquire knowledge about information and improve their oral proficiency.However,research on the relevant oral English autonomous learning ability of English majors shows that the overall learning situation is not satisfying.Based on the development of the concept of autonomous learning,this article explores the current situation and existing problems in oral English autonomous learning of English majors under the context of the Internet,and proposes corresponding autonomous learning strategies for improving their oral English skill.

Evolutionary Decision-Making and Planning for Autonomous Driving Based on Safe and Rational Exploration and Exploitation

Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment.This study proposes an online evolutionary decision-making and motion planning framework for autonomous driving based on a hybrid data-and model-driven method.First,a data-driven decision-making module based on deep reinforcement learning(DRL)is developed to pursue a rational driving performance as much as possible.Then,model predictive control(MPC)is employed to execute both longitudinal and lateral motion planning tasks.Multiple constraints are defined according to the vehicle’s physical limit to meet the driving task requirements.Finally,two principles of safety and rationality for the self-evolution of autonomous driving are proposed.A motion envelope is established and embedded into a rational exploration and exploitation scheme,which filters out unreasonable experiences by masking unsafe actions so as to collect high-quality training data for the DRL agent.Experiments with a high-fidelity vehicle model and MATLAB/Simulink co-simulation environment are conducted,and the results show that the proposed online-evolution framework is able to generate safer,more rational,and more efficient driving action in a real-world environment.

Using TED to Enhance Student Autonomous Learning

Based on the literature review about autonomous learning,the study put forward four steps for using TED to enhance student autonomous learning,which are preparation,activity design,presentation and evaluation. By doing so,both teachers and students can achieve their teaching and learning objectives.

Autonomous Learning and Improving Communicative Competence

Nowadays, English as a world language becomes more and more important. Consequently, English learning becomes more and more popular. As we know, an important object for English learners is to improve their communicative competence. So autonomous learning is a good way to improve communicative competence. In this paper, two terms, autonomous learning and communicative competence, and their relationship will be introduced from the perspective of English learning. Autonomous learning is self-managed learning, which is contrary to passive learning and mechanical learning, according to intrinsic property of language learning. Communicative competence is a concept introduced by Dell Hymes and is discussed and refined by many other linguists. According to Hymes, communicative competence is the ability not only to apply the grammatical rules of language in order to form grammatically correct sentences but also to know when and where to use these sentences and to whom. Communicative competence includes 4 aspects: Possibility, feasibility, appropriateness and performance. Improving communicative competence is the result of autonomous learning, autonomous learning is the motivation of improving communicative competence. English, of course, is a bridge connecting China to the world, and fostering students'communicative competence through autonomous learning is the vital element of improving English learning in China.

Incorporation of Learning Strategies into Web-based Autonomous Listening

The thesis introduces a comparative study of students'autonomous listening practice in a web-based autonomous learning center and the traditional teacher-dominated listening practice in a traditional language lab.The purpose of the study is to find how students'listening strategies differ in these two approaches and thereby to find which one better facilitates students'listening proficiency.

UAV maneuvering decision-making algorithm based on deep reinforcement learning under the guidance of expert experience

Autonomous umanned aerial vehicle(UAV) manipulation is necessary for the defense department to execute tactical missions given by commanders in the future unmanned battlefield. A large amount of research has been devoted to improving the autonomous decision-making ability of UAV in an interactive environment, where finding the optimal maneuvering decisionmaking policy became one of the key issues for enabling the intelligence of UAV. In this paper, we propose a maneuvering decision-making algorithm for autonomous air-delivery based on deep reinforcement learning under the guidance of expert experience. Specifically, we refine the guidance towards area and guidance towards specific point tasks for the air-delivery process based on the traditional air-to-surface fire control methods.Moreover, we construct the UAV maneuvering decision-making model based on Markov decision processes(MDPs). Specifically, we present a reward shaping method for the guidance towards area and guidance towards specific point tasks using potential-based function and expert-guided advice. The proposed algorithm could accelerate the convergence of the maneuvering decision-making policy and increase the stability of the policy in terms of the output during the later stage of training process. The effectiveness of the proposed maneuvering decision-making policy is illustrated by the curves of training parameters and extensive experimental results for testing the trained policy.

On Affective Strategies use in College Students' Autonomous English Learning

Autonomous study emphasizes the learner's initiative,enthusiasm and creativity.In all fields of education,there is growing emphasis on "learner-centered" teaching methods and the ability of learner autonomy.Many experts and scholars have found that learning strategies plays an important role in English language learning,but the importance of affective strategy use in English learning is often ignored by people.Therefore,this paper focuses on the frequencies of affective strategies use in English learning and their relationships so as to enable college students to use positive affective strategies effectively to improve their autonomous learning ability.

A Review of the Effectiveness of Web-based Course with College English Learners' Autonomous Learning

The paper is a literature review, aiming to examine the effectiveness of web-based college English learning which mainly focuses on learners' autonomous learning. Previous studies indicate that the web-based learning can improve learners' autonomous learning, as well as some problems found in their findings. Therefore, this paper first gives a summary and critique of research studies on the web-based autonomous learning and some factors influencing learners' autonomous learning ability;then, areas that deserve further study are also indicated.

Cultivation of Autonomous Learning Ability and Intercultural Communication Competence in Foreign Language Teaching

Learner Autonomy has been a hot topic in foreign language learning and teaching since 1960s,especially in relation to life-long skills.As the globalization develops,intercultural communication becomes more and more significant for college students.This essay attempts to explore main approaches to cultivate and improve students' autonomous learning ability and intercultural communication competence in foreign language teaching.

Deep Imitation Learning for Autonomous Vehicles Based on Convolutional Neural Networks

Providing autonomous systems with an effective quantity and quality of information from a desired task is challenging. In particular, autonomous vehicles, must have a reliable vision of their workspace to robustly accomplish driving functions. Speaking of machine vision, deep learning techniques, and specifically convolutional neural networks, have been proven to be the state of the art technology in the field. As these networks typically involve millions of parameters and elements, designing an optimal architecture for deep learning structures is a difficult task which is globally under investigation by researchers. This study experimentally evaluates the impact of three major architectural properties of convolutional networks, including the number of layers, filters, and filter size on their performance. In this study, several models with different properties are developed,equally trained, and then applied to an autonomous car in a realistic simulation environment. A new ensemble approach is also proposed to calculate and update weights for the models regarding their mean squared error values. Based on design properties,performance results are reported and compared for further investigations. Surprisingly, the number of filters itself does not largely affect the performance efficiency. As a result, proper allocation of filters with different kernel sizes through the layers introduces a considerable improvement in the performance.Achievements of this study will provide the researchers with a clear clue and direction in designing optimal network architectures for deep learning purposes.

Autonomous Maneuver Decisions via Transfer Learning Pigeon-Inspired Optimization for UCAVs in Dogfight Engagements

This paper proposes an autonomous maneuver decision method using transfer learning pigeon-inspired optimization(TLPIO)for unmanned combat aerial vehicles(UCAVs)in dogfight engagements.Firstly,a nonlinear F-16 aircraft model and automatic control system are constructed by a MATLAB/Simulink platform.Secondly,a 3-degrees-of-freedom(3-DOF)aircraft model is used as a maneuvering command generator,and the expanded elemental maneuver library is designed,so that the aircraft state reachable set can be obtained.Then,the game matrix is composed with the air combat situation evaluation function calculated according to the angle and range threats.Finally,a key point is that the objective function to be optimized is designed using the game mixed strategy,and the optimal mixed strategy is obtained by TLPIO.Significantly,the proposed TLPIO does not initialize the population randomly,but adopts the transfer learning method based on Kullback-Leibler(KL)divergence to initialize the population,which improves the search accuracy of the optimization algorithm.Besides,the convergence and time complexity of TLPIO are discussed.Comparison analysis with other classical optimization algorithms highlights the advantage of TLPIO.In the simulation of air combat,three initial scenarios are set,namely,opposite,offensive and defensive conditions.The effectiveness performance of the proposed autonomous maneuver decision method is verified by simulation results.

Reflections on English Listening Autonomous Learning in the Independent College

The self-access learning center emphasizes the principle and significance of student-centeredness rather than teacher-centeredness,which is the focus of the reform of college English in the independent college.To implement belief training program and assist students to grow into proper autonomous learners is the suggestion given when students lack beliefs and motivations.

A learning-based flexible autonomous motion control method for UAV in dynamic unknown environments

This paper presents a deep reinforcement learning(DRL)-based motion control method to provide unmanned aerial vehicles(UAVs)with additional flexibility while flying across dynamic unknown environments autonomously.This method is applicable in both military and civilian fields such as penetration and rescue.The autonomous motion control problem is addressed through motion planning,action interpretation,trajectory tracking,and vehicle movement within the DRL framework.Novel DRL algorithms are presented by combining two difference-amplifying approaches with traditional DRL methods and are used for solving the motion planning problem.An improved Lyapunov guidance vector field(LGVF)method is used to handle the trajectory-tracking problem and provide guidance control commands for the UAV.In contrast to conventional motion-control approaches,the proposed methods directly map the sensorbased detections and measurements into control signals for the inner loop of the UAV,i.e.,an end-to-end control.The training experiment results show that the novel DRL algorithms provide more than a 20%performance improvement over the state-ofthe-art DRL algorithms.The testing experiment results demonstrate that the controller based on the novel DRL and LGVF,which is only trained once in a static environment,enables the UAV to fly autonomously in various dynamic unknown environments.Thus,the proposed technique provides strong flexibility for the controller.

Reinforcement learning based parameter optimization of active disturbance rejection control for autonomous underwater vehicle

This paper proposes a liner active disturbance rejection control(LADRC) method based on the Q-Learning algorithm of reinforcement learning(RL) to control the six-degree-of-freedom motion of an autonomous underwater vehicle(AUV).The number of controllers is increased to realize AUV motion decoupling.At the same time, in order to avoid the oversize of the algorithm, combined with the controlled content, a simplified Q-learning algorithm is constructed to realize the parameter adaptation of the LADRC controller.Finally, through the simulation experiment of the controller with fixed parameters and the controller based on the Q-learning algorithm, the rationality of the simplified algorithm, the effectiveness of parameter adaptation, and the unique advantages of the LADRC controller are verified.

Research on the Construction of English Autonomous Learning Monitoring Mode under the Background of Big Data

The era of big data is coming,the combination of big data and traditional teaching can provide more and more accurate services for students'self-learning,and it is a good way to teach students according to their aptitude.In this background,a learning society is coming,which aiming at learning,autonomous learning and lifelong learning.Learning society emphasize the ability of learning autonomy for students unprecedentedly.Learning is no longer limited to the campus.Learning ability will accompany learners'social life and become an active and healthy lifelong activity.Autonomous learning is a learning theory that goes with the requirements of The Times and has a broad development prospect.The study of Autonomous learning not only has a very important guiding significance for the educational and teaching practice in China,but also plays an important role in the life development of every student.The subject of learning is gradually transferred from the classroom,teachers and textbooks to the students themselves.Teachers should not only impart knowledge and answer questions,but also,most importantly,teach students how to exert their autonomy in autonomous learning.After investigating and researching the existing monitoring model of autonomous English learning in colleges and universities,our group found that in practice,there is a lack of corresponding monitoring mechanisms and means,and autonomous learning has gradually become formalized.Therefore,according to the actual situation of autonomous English learning in our country's universities,the monitoring model of autonomous English learning has been reconstructed,and an effective comprehensive evaluation system has been established to effectively improve students'English learning ability.