While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present...While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present a novel robust reinforcement learning approach with safety guarantees to attain trustworthy decision-making for autonomous vehicles.The proposed technique ensures decision trustworthiness in terms of policy robustness and collision safety.Specifically,an adversary model is learned online to simulate the worst-case uncertainty by approximating the optimal adversarial perturbations on the observed states and environmental dynamics.In addition,an adversarial robust actor-critic algorithm is developed to enable the agent to learn robust policies against perturbations in observations and dynamics.Moreover,we devise a safety mask to guarantee the collision safety of the autonomous driving agent during both the training and testing processes using an interpretable knowledge model known as the Responsibility-Sensitive Safety Model.Finally,the proposed approach is evaluated through both simulations and experiments.These results indicate that the autonomous driving agent can make trustworthy decisions and drastically reduce the number of collisions through robust safety policies.展开更多
As a large amount of data needs to be processed and speed needs to be improved,edge computing with ultra-low latency and ultra-connectivity is emerging as a new paradigm.These changes can lead to new cyber risks,and s...As a large amount of data needs to be processed and speed needs to be improved,edge computing with ultra-low latency and ultra-connectivity is emerging as a new paradigm.These changes can lead to new cyber risks,and should therefore be considered for a security threat model.To this end,we constructed an edge system to study security in two directions,hardware and software.First,on the hardware side,we want to autonomically defend against hardware attacks such as side channel attacks by configuring field programmable gate array(FPGA)which is suitable for edge computing and identifying communication status to control the communication method according to priority.In addition,on the software side,data collected on the server performs end-to-end encryption via symmetric encryption keys.Also,we modeled autonomous defense systems on the server by using machine learning which targets to incoming and outgoing logs.Server log utilizes existing intrusion detection datasets that should be used in real-world environments.Server log was used to detect intrusion early by modeling an intrusion prevention system to identify behaviors that violate security policy,and to utilize the existing intrusion detection data set that should be used in a real environment.Through this,we designed an efficient autonomous defense system that can provide a stable system by detecting abnormal signals from the device and converting them to an effective method to control edge computing,and to detect and control abnormal intrusions on the server side.展开更多
Most real-world situations involve unavoidable measurement noises or perception errors which result in unsafe decision making or even casualty in autonomous driving.To address these issues and further improve safety,a...Most real-world situations involve unavoidable measurement noises or perception errors which result in unsafe decision making or even casualty in autonomous driving.To address these issues and further improve safety,automated driving is required to be capable of handling perception uncertainties.Here,this paper presents an observation-robust reinforcement learning against observational uncertainties to realize safe decision making for autonomous vehicles.Specifically,an adversarial agent is trained online to generate optimal adversarial attacks on observations,which attempts to amplify the average variation distance on perturbed policies.In addition,an observation-robust actor-critic approach is developed to enable the agent to learn the optimal policies and ensure that the changes of the policies perturbed by optimal adversarial attacks remain within a certain bound.Lastly,the safe decision making scheme is evaluated on a lane change task under complex highway traffic scenarios.The results show that the developed approach can ensure autonomous driving performance,as well as the policy robustness against adversarial attacks on observations.展开更多
基金supported in part by the Start-Up Grant-Nanyang Assistant Professorship Grant of Nanyang Technological Universitythe Agency for Science,Technology and Research(A*STAR)under Advanced Manufacturing and Engineering(AME)Young Individual Research under Grant(A2084c0156)+2 种基金the MTC Individual Research Grant(M22K2c0079)the ANR-NRF Joint Grant(NRF2021-NRF-ANR003 HM Science)the Ministry of Education(MOE)under the Tier 2 Grant(MOE-T2EP50222-0002)。
文摘While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present a novel robust reinforcement learning approach with safety guarantees to attain trustworthy decision-making for autonomous vehicles.The proposed technique ensures decision trustworthiness in terms of policy robustness and collision safety.Specifically,an adversary model is learned online to simulate the worst-case uncertainty by approximating the optimal adversarial perturbations on the observed states and environmental dynamics.In addition,an adversarial robust actor-critic algorithm is developed to enable the agent to learn robust policies against perturbations in observations and dynamics.Moreover,we devise a safety mask to guarantee the collision safety of the autonomous driving agent during both the training and testing processes using an interpretable knowledge model known as the Responsibility-Sensitive Safety Model.Finally,the proposed approach is evaluated through both simulations and experiments.These results indicate that the autonomous driving agent can make trustworthy decisions and drastically reduce the number of collisions through robust safety policies.
基金This research was funded by Korea Environmental Industry&Technology Institute(KEITI),Grant Number RE202101551and The APC was funded by Ministry of Environment(ME).
文摘As a large amount of data needs to be processed and speed needs to be improved,edge computing with ultra-low latency and ultra-connectivity is emerging as a new paradigm.These changes can lead to new cyber risks,and should therefore be considered for a security threat model.To this end,we constructed an edge system to study security in two directions,hardware and software.First,on the hardware side,we want to autonomically defend against hardware attacks such as side channel attacks by configuring field programmable gate array(FPGA)which is suitable for edge computing and identifying communication status to control the communication method according to priority.In addition,on the software side,data collected on the server performs end-to-end encryption via symmetric encryption keys.Also,we modeled autonomous defense systems on the server by using machine learning which targets to incoming and outgoing logs.Server log utilizes existing intrusion detection datasets that should be used in real-world environments.Server log was used to detect intrusion early by modeling an intrusion prevention system to identify behaviors that violate security policy,and to utilize the existing intrusion detection data set that should be used in a real environment.Through this,we designed an efficient autonomous defense system that can provide a stable system by detecting abnormal signals from the device and converting them to an effective method to control edge computing,and to detect and control abnormal intrusions on the server side.
基金supported by Foundation of State Key Laboratory of Automotive Simulation and Control.
文摘Most real-world situations involve unavoidable measurement noises or perception errors which result in unsafe decision making or even casualty in autonomous driving.To address these issues and further improve safety,automated driving is required to be capable of handling perception uncertainties.Here,this paper presents an observation-robust reinforcement learning against observational uncertainties to realize safe decision making for autonomous vehicles.Specifically,an adversarial agent is trained online to generate optimal adversarial attacks on observations,which attempts to amplify the average variation distance on perturbed policies.In addition,an observation-robust actor-critic approach is developed to enable the agent to learn the optimal policies and ensure that the changes of the policies perturbed by optimal adversarial attacks remain within a certain bound.Lastly,the safe decision making scheme is evaluated on a lane change task under complex highway traffic scenarios.The results show that the developed approach can ensure autonomous driving performance,as well as the policy robustness against adversarial attacks on observations.
文摘针对城市场景下巡飞弹自主协同饱和攻击问题,将其建模为分布式部分可观测马尔可夫决策过程(Dec-POMDPs),设计了确保巡飞弹在极小时间间隔内到达的专用奖励函数,并结合使用联合权重参数的奖励函数,采用循环多智能体深度确定性策略梯度算法(R-MADDPG)训练巡飞弹自主协同饱和攻击策略,使用蒙特卡罗方法分析指标成功率.仿真实验结果表明,在训练后的决策模型引导下,巡飞弹执行自主协同饱和攻击的任务成功率为93.2%,其中,机间避撞率为94.4%、空中突防成功率为99.5%,95.3%回合到达最大时间间隔小于0.4 s.
