Toward Trustworthy Decision-Making for Autonomous Vehicles:A Robust Reinforcement Learning Approach with Safety Guarantees

While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present a novel robust reinforcement learning approach with safety guarantees to attain trustworthy decision-making for autonomous vehicles.The proposed technique ensures decision trustworthiness in terms of policy robustness and collision safety.Specifically,an adversary model is learned online to simulate the worst-case uncertainty by approximating the optimal adversarial perturbations on the observed states and environmental dynamics.In addition,an adversarial robust actor-critic algorithm is developed to enable the agent to learn robust policies against perturbations in observations and dynamics.Moreover,we devise a safety mask to guarantee the collision safety of the autonomous driving agent during both the training and testing processes using an interpretable knowledge model known as the Responsibility-Sensitive Safety Model.Finally,the proposed approach is evaluated through both simulations and experiments.These results indicate that the autonomous driving agent can make trustworthy decisions and drastically reduce the number of collisions through robust safety policies.

Results of the virtual simulation teaching method on autonomous learning competencies of undergraduate nursing students

Background:With the continuous development of information technology,most universities use mobile teaching platforms for classroom teaching.With the help of the national virtual simulation experimental teaching project sharing platform,students can enhance self-directed learning through the virtual simulation operations of the project.Purpose:To explore the application of virtual simulation experiment in teaching the fundamentals of nursing practice based on the Platform of the National Virtual Simulation Experiment Teaching Project during the COVID-19 pandemic analyze the impact of this teaching method on the autonomous learning ability of undergraduate nursing students.Methods:Convenience sampling was used to select 121 nursing undergraduates from Y University’s School of Nursing;the online teaching of fundamentals of nursing practice was conducted to the students.After taking the course,questionnaires were distributed to the undergraduate nursing students to collect their perceptions regarding the use of the virtual simulation experiment platform and autonomous learning competencies.Results:Most students expressed their preference for the virtual simulation teaching platform,and their satisfaction with the project evaluation was high 83.05%.They hoped to promote the application in future experimental teaching.Undergraduate nursing students believed that the virtual simulation teaching platform was conducive to cultivating clinical thinking ability,could stimulate learning interest,enhanced autonomous learning competencies.Conclusion:During the pandemic,the virtual simulation teaching platform for a lecture on in nursing education has achieved good results in both the aspects of teaching and student learning.Teachers efficiently used their training time and reduced their teaching burden.Moreover,the laboratory cost was also reduced.For undergraduate nursing students,the system was conducive to cultivating clinical thinking ability,stimulating their interest in learning,enhancing their learning and comprehension abilities and learning initiative.

Autonomous Vehicle Platoons In Urban Road Networks:A Joint Distributed Reinforcement Learning and Model Predictive Control Approach

In this paper, platoons of autonomous vehicles operating in urban road networks are considered. From a methodological point of view, the problem of interest consists of formally characterizing vehicle state trajectory tubes by means of routing decisions complying with traffic congestion criteria. To this end, a novel distributed control architecture is conceived by taking advantage of two methodologies: deep reinforcement learning and model predictive control. On one hand, the routing decisions are obtained by using a distributed reinforcement learning algorithm that exploits available traffic data at each road junction. On the other hand, a bank of model predictive controllers is in charge of computing the more adequate control action for each involved vehicle. Such tasks are here combined into a single framework:the deep reinforcement learning output(action) is translated into a set-point to be tracked by the model predictive controller;conversely, the current vehicle position, resulting from the application of the control move, is exploited by the deep reinforcement learning unit for improving its reliability. The main novelty of the proposed solution lies in its hybrid nature: on one hand it fully exploits deep reinforcement learning capabilities for decisionmaking purposes;on the other hand, time-varying hard constraints are always satisfied during the dynamical platoon evolution imposed by the computed routing decisions. To efficiently evaluate the performance of the proposed control architecture, a co-design procedure, involving the SUMO and MATLAB platforms, is implemented so that complex operating environments can be used, and the information coming from road maps(links,junctions, obstacles, semaphores, etc.) and vehicle state trajectories can be shared and exchanged. Finally by considering as operating scenario a real entire city block and a platoon of eleven vehicles described by double-integrator models, several simulations have been performed with the aim to put in light the main f eatures of the proposed approach. Moreover, it is important to underline that in different operating scenarios the proposed reinforcement learning scheme is capable of significantly reducing traffic congestion phenomena when compared with well-reputed competitors.

A Combined Reinforcement Learning and Model Predictive Control for Car-Following Maneuver of Autonomous Vehicles

Model predictive control is widely used in the design of autonomous driving algorithms.However,its parameters are sensitive to dynamically varying driving conditions,making it difficult to be implemented into practice.As a result,this study presents a self-learning algorithm based on reinforcement learning to tune a model predictive controller.Specifically,the proposed algorithm is used to extract features of dynamic traffic scenes and adjust the weight coefficients of the model predictive controller.In this method,a risk threshold model is proposed to classify the risk level of the scenes based on the scene features,and aid in the design of the reinforcement learning reward function and ultimately improve the adaptability of the model predictive controller to real-world scenarios.The proposed algorithm is compared to a pure model predictive controller in car-following case.According to the results,the proposed method enables autonomous vehicles to adjust the priority of performance indices reasonably in different scenarios according to risk variations,showing a good scenario adaptability with safety guaranteed.

Autonomous air combat decision-making of UAV based on parallel self-play reinforcement learning

Aiming at addressing the problem of manoeuvring decision-making in UAV air combat,this study establishes a one-to-one air combat model,defines missile attack areas,and uses the non-deterministic policy Soft-Actor-Critic(SAC)algorithm in deep reinforcement learning to construct a decision model to realize the manoeuvring process.At the same time,the complexity of the proposed algorithm is calculated,and the stability of the closed-loop system of air combat decision-making controlled by neural network is analysed by the Lyapunov function.This study defines the UAV air combat process as a gaming process and proposes a Parallel Self-Play training SAC algorithm(PSP-SAC)to improve the generalisation performance of UAV control decisions.Simulation results have shown that the proposed algorithm can realize sample sharing and policy sharing in multiple combat environments and can significantly improve the generalisation ability of the model compared to independent training.

Deep Learning Based Autonomous Transport System for Secure Vehicle and Cargo Matching

The latest 6G improvements secured autonomous driving's realism in Intelligent Autonomous Transport Systems(IATS).Despite the IATS's benefits,security remains a significant challenge.Blockchain technology has grown in popularity as a means of implementing safe,dependable,and decentralised independent IATS systems,allowing for more utilisation of legacy IATS infrastructures and resources,which is especially advantageous for crowdsourcing technologies.Blockchain technology can be used to address security concerns in the IATS and to aid in logistics development.In light of the inadequacy of reliance and inattention to rights created by centralised and conventional logistics systems,this paper discusses the creation of a blockchain-based IATS powered by deep learning for secure cargo and vehicle matching(BDL-IATS).The BDL-IATS approach utilises Ethereum as the primary blockchain for storing private data such as order and shipment details.Additionally,the deep belief network(DBN)model is used to select suitable vehicles and goods for transportation.Additionally,the chaotic krill herd technique is used to tune the DBN model’s hyper-parameters.The performance of the BDL-IATS technique is validated,and the findings are inspected under a variety of conditions.The simulationfindings indicated that the BDL-IATS strategy outperformed recent state-of-the-art approaches.

Autonomous machine learning for early bot detection in the internet of things

The high costs incurred due to attacks and the increasing number of different devices in the Internet of Things(IoT)highlight the necessity of the early detection of botnets(i.e.,a network of infected devices)to gain an advantage against attacks.However,early botnet detection is challenging because of continuous malware mutations,the adoption of sophisticated obfuscation techniques,and the massive volume of data.The literature addresses botnet detection by modeling the behavior of malware spread,the classification of malicious traffic,and the analysis of traffic anomalies.This article details ANTE,a system for ANTicipating botnEt signals based on machine learning algorithms.The system adapts itself to different scenarios and detects different types of botnets.It autonomously selects the most appropriate Machine Learning(ML)pipeline for each botnet and improves the classification before an attack effectively begins.The system evaluation follows trace-driven experiments and compares ANTE results to other relevant results from the literature over four representative datasets:ISOT HTTP Botnet,CTU-13,CICDDoS2019,and BoT-IoT.Results show an average detection accuracy of 99.06%and an average bot detection precision of 100%.

Autonomous landing scene recognition based on transfer learning for drones

In this paper, we study autonomous landing scene recognition with knowledge transfer for drones. Considering the difficulties in aerial remote sensing, especially that some scenes are extremely similar, or the same scene has different representations in different altitudes, we employ a deep convolutional neural network(CNN) based on knowledge transfer and fine-tuning to solve the problem. Then, LandingScenes-7 dataset is established and divided into seven classes. Moreover, there is still a novelty detection problem in the classifier, and we address this by excluding other landing scenes using the approach of thresholding in the prediction stage. We employ the transfer learning method based on ResNeXt-50 backbone with the adaptive momentum(ADAM) optimization algorithm. We also compare ResNet-50 backbone and the momentum stochastic gradient descent(SGD) optimizer. Experiment results show that ResNeXt-50 based on the ADAM optimization algorithm has better performance. With a pre-trained model and fine-tuning, it can achieve 97.845 0% top-1 accuracy on the LandingScenes-7dataset, paving the way for drones to autonomously learn landing scenes.

An Investigation of College English Autonomous Learning in Network Multimodal Context

In the current society, based on the growing development of network information technology, the teaching in many colleges and universities has also introduced it to adapt to the situation. This trend can provide more useful conditions for students to learn, which requires students to master enough self-learning abilities to adapt to this model. The study in the paper shows that students are usually interested in autonomous learning in a multimodal environment, but the degree of strategy choice is relatively low, and the learning process is blind and passive with the lack of self-confidence. Facing the future, schools should actively integrate into network thinking, and teachers should change their roles and train and guide students’ learning strategies and learning motivations, so as to achieve better teaching results.

Research on College Studentst'Autonomous EFL Learning Affer Course Exemption

By analyzing the English learning logs of 12 students in a provincial university in south-west China after they had been exempted from taking college English courses,this study investigated college students’autonomous EFL(English as a foreign language)learning after course exemption,including the use of mediational means in EFL learning,EFL learning hours,and other factors affecting EFL learning,in the hope of giving new perspectives on college ELF curriculum design,teaching,and education management.

Model-Free Formation Control of Autonomous Underwater Vehicles:A Broad Learning-Based Solution

Dear Editor,We develop a broad learning-based algorithm to enforce the formation control of AUVs.Compared with the deep learning(DL)based formation solutions,our solution employs the broad learning system(BLS)to remodel the learning framework without a retraining process.

Oral English Autonomous Learning Ability of English Majors Under the Background of the Internet and Its Improvement

The Internet is an important means of communication for contemporary college students,especially those majoring in English,to acquire knowledge about information and improve their oral proficiency.However,research on the relevant oral English autonomous learning ability of English majors shows that the overall learning situation is not satisfying.Based on the development of the concept of autonomous learning,this article explores the current situation and existing problems in oral English autonomous learning of English majors under the context of the Internet,and proposes corresponding autonomous learning strategies for improving their oral English skill.

基于改进Q-Learning的移动机器人路径规划算法

随着移动机器人在生产生活中的深入应用,其路径规划能力也需要向快速性和环境适应性兼备发展。为解决现有移动机器人使用强化学习方法进行路径规划时存在的探索前期容易陷入局部最优、反复搜索同一区域,探索后期收敛率低、收敛速度慢的问题,本研究提出一种改进的Q-Learning算法。该算法改进Q矩阵赋值方法,使迭代前期探索过程具有指向性,并降低碰撞的情况;改进Q矩阵迭代方法,使Q矩阵更新具有前瞻性,避免在一个小区域中反复探索;改进随机探索策略,在迭代前期全面利用环境信息,后期向目标点靠近。在不同栅格地图仿真验证结果表明,本文算法在Q-Learning算法的基础上,通过上述改进降低探索过程中的路径长度、减少抖动并提高收敛的速度,具有更高的计算效率。

M-learning结合CBL在消化科规培教学中的探讨及应用

目的探究移动学习平台(M-learning,ML)结合案例教学(Case-based Learning,CBL)在消化科住院医师规范化培训(简称规培)教学中的应用效果。方法选取2021年1月—2023年1月于上海市徐汇区中心医院消化科参加规培学习的80名医师作为研究对象,将其按照随机数表法分为研究组和对照组,每组40名。对照组给予传统讲授式教学法,研究组给予M-learning结合CBL教学法,对比两组医师的理论考试成绩、实践技能考试成绩和学习满意度。结果研究组的理论成绩和实践技能考试成绩均高于对照组,差异具有统计学意义(P均<0.05);研究组的学习满意度明显高于对照组,差异具有统计学意义(P<0.05)。结论将Mlearning结合CBL教学法应用于消化科规培教学中,不仅能够提升医师的理论考试成绩和实践技能考试成绩,还能够有效提高医师学习满意度。

基于Q-Learning的航空器滑行路径规划研究

针对传统算法规划航空器滑行路径准确度低、不能根据整体场面运行情况进行路径规划的问题,提出一种基于Q-Learning的路径规划方法。通过对机场飞行区网络结构模型和强化学习的仿真环境分析,设置了状态空间和动作空间,并根据路径的合规性和合理性设定了奖励函数,将路径合理性评价值设置为滑行路径长度与飞行区平均滑行时间乘积的倒数。最后,分析了动作选择策略参数对路径规划模型的影响。结果表明,与A*算法和Floyd算法相比,基于Q-Learning的路径规划在滑行距离最短的同时,避开了相对繁忙的区域,路径合理性评价值高。

Evolutionary Decision-Making and Planning for Autonomous Driving Based on Safe and Rational Exploration and Exploitation

Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment.This study proposes an online evolutionary decision-making and motion planning framework for autonomous driving based on a hybrid data-and model-driven method.First,a data-driven decision-making module based on deep reinforcement learning(DRL)is developed to pursue a rational driving performance as much as possible.Then,model predictive control(MPC)is employed to execute both longitudinal and lateral motion planning tasks.Multiple constraints are defined according to the vehicle’s physical limit to meet the driving task requirements.Finally,two principles of safety and rationality for the self-evolution of autonomous driving are proposed.A motion envelope is established and embedded into a rational exploration and exploitation scheme,which filters out unreasonable experiences by masking unsafe actions so as to collect high-quality training data for the DRL agent.Experiments with a high-fidelity vehicle model and MATLAB/Simulink co-simulation environment are conducted,and the results show that the proposed online-evolution framework is able to generate safer,more rational,and more efficient driving action in a real-world environment.

Machine learning applications in stroke medicine:advancements,challenges,and future prospectives

Stroke is a leading cause of disability and mortality worldwide,necessitating the development of advanced technologies to improve its diagnosis,treatment,and patient outcomes.In recent years,machine learning techniques have emerged as promising tools in stroke medicine,enabling efficient analysis of large-scale datasets and facilitating personalized and precision medicine approaches.This abstract provides a comprehensive overview of machine learning’s applications,challenges,and future directions in stroke medicine.Recently introduced machine learning algorithms have been extensively employed in all the fields of stroke medicine.Machine learning models have demonstrated remarkable accuracy in imaging analysis,diagnosing stroke subtypes,risk stratifications,guiding medical treatment,and predicting patient prognosis.Despite the tremendous potential of machine learning in stroke medicine,several challenges must be addressed.These include the need for standardized and interoperable data collection,robust model validation and generalization,and the ethical considerations surrounding privacy and bias.In addition,integrating machine learning models into clinical workflows and establishing regulatory frameworks are critical for ensuring their widespread adoption and impact in routine stroke care.Machine learning promises to revolutionize stroke medicine by enabling precise diagnosis,tailored treatment selection,and improved prognostication.Continued research and collaboration among clinicians,researchers,and technologists are essential for overcoming challenges and realizing the full potential of machine learning in stroke care,ultimately leading to enhanced patient outcomes and quality of life.This review aims to summarize all the current implications of machine learning in stroke diagnosis,treatment,and prognostic evaluation.At the same time,another purpose of this paper is to explore all the future perspectives these techniques can provide in combating this disabling disease.

改进Q-Learning的路径规划算法研究

针对Q-Learning算法学习效率低、收敛速度慢且在动态障碍物的环境下路径规划效果不佳的问题,本文提出一种改进Q-Learning的移动机器人路径规划算法.针对该问题,算法根据概率的突变性引入探索因子来平衡探索和利用以加快学习效率;通过在更新函数中设计深度学习因子以保证算法探索概率;融合遗传算法,避免陷入局部路径最优同时按阶段探索最优迭代步长次数,以减少动态地图探索重复率;最后提取输出的最优路径关键节点采用贝塞尔曲线进行平滑处理,进一步保证路径平滑度和可行性.实验通过栅格法构建地图,对比实验结果表明,改进后的算法效率相较于传统算法在迭代次数和路径上均有较大优化,且能够较好的实现动态地图下的路径规划,进一步验证所提方法的有效性和实用性.

基于Q-learning的自适应链路状态路由协议

针对大规模无人机自组网面临的任务需求多样性、电磁环境复杂性、节点高机动性等问题,充分考虑无人机节点高速移动的特点,基于无人机拓扑稳定度和链路通信容量指标设计了一种无人机多点中继(multi-point relay,MPR)选择方法;为了减少网络路由更新时间,增加无人机自组网路由策略的稳定性和可靠性,提出了一种基于Q-learning的自适应链路状态路由协议(Q-learning based adaptive link state routing,QALSR)。仿真结果表明,所提算法性能指标优于现有的主动路由协议。

College English autonomous teaching and learning research and practice

This paper intends to promote a college English autonomous teaching and learning approach by introducing the whole process of its implementation and feedback from the learners. The theoretical and practical framework of this approach is： with multiple autonomous learning research and practice models as its core, with process syllabus as its guidance, with task-based teaching as its essential principle, with group cooperation and reciprocal learning as its means, with extracurricular activities, online learning and self-access center as its learning environment, with formative assessment system as its guarantee and with cultivation of learners＇ comprehensive English practical skills and autonomy as its goal. Through this approach, we provide the learners with a favorable learning environment where they can learn by themselves and learn by reflection and practice so that they can learn how to learn and how to behave and how to survive.