结合图像识别的移动增强现实系统设计与应用

Mobile augmented reality system design and application based on image recognition

目的在移动互联网时代下,移动增强现实应用得到越来越快的发展。然而户外场景中存在许多相似结构的建筑,且手机的存储和计算能力有限,因此应用多集中于室内小范围环境,对于室外大规模复杂场景的适应性较弱。对此,建立一套基于云端图像识别的移动增强现实系统。方法为解决相似特征的误匹配问题,算法中将重力信息加入到SURF和BRISK特征描述中去,构建Gravity-SURF和Gravity-BRISK特征描述。云端系统对增强信息进行有效管理,采用基于Gravity-SURF特征的VLAD方法对大规模图像进行识别;在智能终端上的应用中呈现识别图像的增强信息,并利用识别图像的Gravity-BRISK特征和光流结合的方法对相机进行跟踪,采用Unity3D渲染引擎实时绘制3维模型。结果在包含重力信息的4 000幅户外图像的数据库中进行实验。采用结合重力信息的特征描述算法,能够增强具有相似特征的描述符的区分性,并提高匹配正确率。图像识别算法的识别率能达到88%以上,识别时间在420 ms左右;光流跟踪的RMS误差小于1.2像素,帧率能达到23帧/s。结论本文针对室外大规模复杂场景建立的基于图像识别的移动增强现实系统,能方便对不同应用的增强现实数据进行管理。系统被应用到谷歌眼镜和新闻领域上,不局限于单一的应用领域。结果表明,识别算法和跟踪注册算法能够满足系统的精度和实时性要求。

Objective With the development of mobile Interuet, the speed of advancement of mobile augmented reality ap- plications is accelerating. However, given that storage and computation capacity are limited on mobile devices and that large-scale outdoor images mostly include similar building structure images, these applications are restricted to indoor envi- ronments. To solve this problem, a mobile augmented reality system based on cloud image recognition is established. Method To reduce mistake-matching probability, we add gTavity information into the SURF and BRISK feature description algorithm. Gravity information is used as the main orientation. In our system, these two description algorithms are named Gravity-SURF and Gravity-BRISK. In the cloud, large-scale images and augmented information are managed by the aug- mented reality management system. The VLAD algorithm based on Gravity-SURF is applied to large-scale image recogni- tion. On intelligent terminals, different user interfaces are designed for augmented reality applications to display augmented information. The model rendered is a presentation format of the augmented information. We propose a method that combines Gravity-BRISK and optical flow to perform a real-time camera tracking algorithm. When using rendering engine Unity3D for 3D rendering, we propose a solution for the transformation of camera pose in different coordinate systems. Thereafter, the camera pose data will be transmitted from an Android platform to Unity3D by the Android Native Development Kit. Result In the experiments, we first build an image database with 4 000 images. The database contains 800 outdoor scenes and each scene contains 5 images in different angles and distances. Each image must be associated with the local gravity information. Thereafter, we compare the traditional description algorithm with the gravity-based description algorithm. From the experi- ments, we can conclude that the gravity-based description algorithm is able to enhance the discrimination of similar features and improve the matching accuracy. Finally, we test the performance and accuracy of our image recognition algorithm and tracking algorithm. The recognition algorithm experiments show that the VLAD algorithm based on Gravity-SURF can im- prove the recognition rate. In 4G and WIFI network environments, the upload time of the recognition image is less than 40 ms, the online recognition time is approximately 420 ms, and the augmented data download time is less than 4 s. Fur- thermore, the tracking algorithm experimerits show that the RMS errors in variation of the rotation and scale are all less than 1. 2 pixels. For the optical flow stage, the frame rate is able to reach 23 frame/s. In terms of the above experiments, the algorithm can satisfy the accuracy and real-time demands of the system. Conclusion The mobile augmented reality system is suitable for complex outdoor environments. This system has beela applied in Google Glass and the journalism field but is not limited to these two areas. Nonetheless, the augmented information in different applications can be managed efficiently. The system we established aims to promote the development of MAR applications.