基于临场感增强的果园机器人遥操作可视化系统研究

Teleoperation Visualization System of Orchard Robot Based on Enhancing Telepresence

针对果园作业机器人使用单目相机进行遥操作时,仅用二维视频获取环境信息缺乏临场感的问题,设计了一套基于临场感增强的果园环境信息可视化系统,用于果园机器人遥操作。系统由计算服务器、云服务器、网络摄像头、激光雷达、嵌入式开发平台等组成。计算服务器采用T7920工作站,并在其上部署Tensorflow计算框架和Open3D点云算法库,计算服务器在接收到云服务器转发来的环境图像和点云数据后,分别对图像进行导航信息增强,对点云进行曲面重建;嵌入式开发平台可以收集来自于网络摄像头和激光雷达的原始数据,并上传至云服务器;在云服务器部署了以ZeroMQ为基础的消息中转程序和HTML5后台服务,提供跨互联网的消息通信服务和可移动的遥操作环境信息可视化服务。测试结果表明,部署在计算服务器的导航信息提取模型平均提取导航线时间86 ms,提取导航线平均精度16°,均优于对比模型结果。点云重建算法可以有效建立场景轮廓,平均精度4.9 cm,平均重建时间24 ms。压缩图像传输及增强处理时延不超过230 ms,点云的传输时延不超过400 ms。各项参数可以满足遥操作机器人进行果园作业的基本要求,相比仅有单目相机的遥操作,临场感明显增强,可为果园机器人遥操作提供参考。

Aiming at the problem of lack of presence when the orchard robot used a monocular camera for remote operation,only using two-dimensional video to obtain environmental information lacked presence,a set of orchard environment information visualization system based on the enhanced sense of presence was designed.The system consisted of computing server,cloud server,network camera,Li DAR,embedded development platform,etc.The computing server adopted the T7920 workstation,and deployed the Tensorflow computing framework and the Open3D algorithm library of point cloud on it.After receiving the environmental image and point cloud data forwarded by the cloud server,the computing server enhanced the navigation information of the image,and surface reconstructed the point cloud.The embedded development platform could collect raw data from webcam and Li DAR,and uploaded them to cloud servers.A ZeroMQ-based message transfer program and HTML5 background service were deployed on the cloud server,providing cross-Internet message communication services and mobile teleoperation environment information visualization services.The test results showed that the average extraction time of the extraction model for navigation information deployed on the computing server was 86 ms,and the average precision of the navigation line extraction was 16°,which were better than the results of the comparison model.The algorithm of point cloud reconstruction can effectively establish scene contours with an average accuracy of 4.9 cm and an average reconstruction time of 24 ms.The delay of compressed image transmission and enhancement processing did not exceed 230 ms,and the transmission delay of point cloud did not exceed 400 ms.The parameters could meet the basic requirements of the remote operation robot for orchard operation.The system significantly enhanced telepresence compared with that only with monocular cameras,which provided an effective reference for the remote operation of the orchard robot.