基于多模态中间表示的端到端自动驾驶模型

End-to-end autonomous driving model based on multi-modal intermediate representations

对驾驶环境的准确理解是实现自动驾驶的先决条件之一。为提高自动驾驶车辆的场景理解能力,提出了一种基于语义分割、水平视差和角度编码的多模态中间表示的端到端自动驾驶模型。该端到端自动驾驶模型利用深度学习技术构建感知-规划网络。感知网络以RGB和深度图为输入生成多模态中间表示,实现道路环境及周围障碍物的空间分布描述;规划网络使用多模态中间表示进行道路环境特征提取和航路点预测。基于CARLA仿真平台进行模型的训练和性能测试,结果表明:该端到端自动驾驶模型能够实现对城市道路环境的场景理解,有效地减少了碰撞;相较于单模态中间表示的基线模型,其驾驶性能指标提升了31.47%。

An accurate understanding of the driving environment is one of the prerequisites for autonomous driving.In order to im-prove the scene understanding ability of autonomous driving vehicles,an end-to-end autonomous driving model based on semantic segmentation,horizontal disparity,and angle coding multi-modal intermediate representations was proposed.The end-to-end auton-omous driving model used deep learning technology to build perception-planning network.The perception network generated multi-modal intermediate representations with RGB and depth images as inputs to realize the spatial distribution description of road en-vironment and surrounding obstacles.The planning network used multi-modal intermediate representations to extract road environ-ment features and predict waypoints.Model training and performance testing were conducte based on the CARLA simulation plat-form.The results showed that the end-to-end autonomous driving model can realize the scene understanding of urban road environ-ment and effectively reduce collisions.Compared with the baseline model based on the single modal intermediate representation,its driving performance index is 31.47%better than the baseline model.