The recent fast development in computer vision and mobile sensor technology such as mobile LiDAR and RGB-D cameras is pushing the boundary of the technology to suit the need of real-life applications in the fields of ...The recent fast development in computer vision and mobile sensor technology such as mobile LiDAR and RGB-D cameras is pushing the boundary of the technology to suit the need of real-life applications in the fields of Augmented Reality(AR),robotics,indoor GIS and self-driving.Camera localization is often a key and enabling technology among these applications.In this paper,we developed a novel camera localization workflow based on a highly accurate 3D prior map optimized by our RGBD SLAM method in conjunction with a deep learning routine trained using consecutive video frames labeled with high precision camera pose.Furthermore,an AR registration method tightly coupled with a game engine is proposed,which incorporates the proposed localization algorithm and aligns the real Kinetic camera with a virtual camera of the game engine to facilitate AR application development in an integrated manner.The experimental results show that the localization accuracy can achieve an average error of 35 cm based on a fine-tuned prior 3D feature database at 3 cm accuracy compared against the ground-truth 3D LiDAR map.The influence of the localization accuracy on the visual effect of AR overlay is also demonstrated and the alignment of the real and virtual camera streamlines the implementation of AR fire emergency response demo in a Virtual Geographic Environment.展开更多
基金This work was funded by the National Key Research and Development Program of China[grant number 2016YFB0502102]It was also partially funded by National Natural Science Foundation of China[grant number 41101436]the Scientific Research Foundation for the Returned Overseas Chinese Scholars,State Education Ministry。
文摘The recent fast development in computer vision and mobile sensor technology such as mobile LiDAR and RGB-D cameras is pushing the boundary of the technology to suit the need of real-life applications in the fields of Augmented Reality(AR),robotics,indoor GIS and self-driving.Camera localization is often a key and enabling technology among these applications.In this paper,we developed a novel camera localization workflow based on a highly accurate 3D prior map optimized by our RGBD SLAM method in conjunction with a deep learning routine trained using consecutive video frames labeled with high precision camera pose.Furthermore,an AR registration method tightly coupled with a game engine is proposed,which incorporates the proposed localization algorithm and aligns the real Kinetic camera with a virtual camera of the game engine to facilitate AR application development in an integrated manner.The experimental results show that the localization accuracy can achieve an average error of 35 cm based on a fine-tuned prior 3D feature database at 3 cm accuracy compared against the ground-truth 3D LiDAR map.The influence of the localization accuracy on the visual effect of AR overlay is also demonstrated and the alignment of the real and virtual camera streamlines the implementation of AR fire emergency response demo in a Virtual Geographic Environment.
