面向智能驾驶的深度学习综合实验平台构建及项目实现

Autonomous driving-oriented construction of comprehensive experimental platform for deep learning and project implementation

搭建了面向智能驾驶的深度学习综合实验平台。平台包括基于Coppelia Sim的智能驾驶虚拟仿真系统和基于英伟达Jetson Nano嵌入式小车的交通沙盘实物系统。基于该平台设计的实验方案,涉及了深度学习主要知识点,实现了智能驾驶若干子场景,采用了边缘人工智能常用方案“云端训练+边缘推断”。基于上述虚拟仿真系统,建模并训练实现了深度强化学习Q值网络、卷积神经网络、循环神经网络等多种类别神经网络。实物系统测试结果表明,该平台基于上述神经网络能够有效实现车道保持、交通标志牌识别、语音控制等智能驾驶场景。学生可利用该平台自主搭建需要的场景,接入不同类型的神经网络,实现其他智能驾驶功能,并完成深度学习相关算法的测试和评估。

[Objective]Deep learning–based autonomous driving has attracted tremendous research interest nowadays.This work aims to design a platform for implementing various deep learning algorithms and applying them in autonomous driving scenarios.To demonstrate the popular“cloud-training and edge inferring”concept,a virtual simulation system and a real traffic and table running system should be designed in the platform.Students are expected to be able to set up the simulation environment for several autonomous driving tasks in the virtual system,including virtual cars,neural networks,and traffic systems.Then,the neural networks for various tasks should be trained and optimized by the students in the virtual system.Finally,the real traffic sand table system should provide model cars,sensors,lanes,and embedded chips for the students to deploy the trained neural networks and test those typical autonomous driving scenarios.[Methods]In the virtual simulation system,three typical deep learning methods and the associated neural networks,namely,deep reinforcement learning,deep convolution,and recursive neural networks,were designed,aiming at three typical autonomous driving scenarios:lane keeping,traffic sign recognition,and voice control.Here,the widely used CoppeliaSim®was employed for constructing the virtual simulation system,and Autodesk was exploited to make traffic signs and other things in the virtual system.In the real traffic sand table system,a toy car with a JetsonNano-embedded chip was bought,and a 4-m×2-m sand table system mimicking the traffic campus of Huazhong University of Science and Technology was manufactured.The communication software ZeroMQ was then employed as the main controller from which the motion control information for the car engine was generated and to which the decisions made by various deep learning neural networks under various driving scenarios were sent.The experimental demonstration for edge intelligence was designed following the“cloud-training and edge inferring”rule:powerful computational servers were employed to train deep learning neural networks for different autonomous driving tasks in the virtual system,and the trained neural networks were then written into the JetsonNano chip embedded in the toy car.In the real traffic sand table system,OpenCV was employed to read the real-time information obtained from a car camera and to transfer it to the neural network,whereas microphones were loaded on the car to listen to possible voice instructions and to send them to the neural networks.By dealing with the visual and audial information from the car sensors,neural networks make decisions on the car's motion,and the car control system follows these real-time instructions.[Results]1)A comprehensive practice platform for deep learning aiming at autonomous driving was built,including a virtual simulation system of autonomous driving based on CoppeliaSim and a physical system of traffic sand table together with an NVIDIA JetsonNano-embedded car.2)Based on the platform,an experimental scheme covering the major knowledge points of deep learning and several subscenarios of autonomous driving was designed.3)Based on the developed virtual simulation system,the deep reinforcement learning Q-value network,convolutional neural network,recurrent neural network,and other types of neural networks were modeled and trained.[Conclusions]Aiming at the central knowledge points of deep learning,an autonomous driving-oriented experimental platform was designed and realized.A virtual simulation system and a real traffic sand table were integrated into the platform,implementing the“cloud-training and edge inferring”tasks,respectively.The tackling of three typical driving scenarios known as lane keeping,traffic sign recognition,and voice control was demonstrated using deep reinforcement and convolution and recursive neural networks based on this developed platform.