移动在线实时绘制技术研究综述

A review of real-time rendering technology based on mobile internet platforms

移动在线实时绘制技术受移动互联网发展的驱动,为3维可视化、计算机视觉、虚拟现实、增强现实、扩展现实和元宇宙等新兴研究领域提供了核心技术的支撑。本文以在线实时绘制技术为切入点,探讨了该技术在移动端、Web端、云端和多端协同这4类平台下的发展重心和研究现状,并深度阐述了工业级在线云平台的实施方案。首先,针对移动端的在线实时绘制,分析了近年来移动端绘制硬件构架设计的优化方向;探讨了在功耗和带宽受到制约的情况下移动端如何对渲染算法进行加速,如何对高功耗的光线跟踪算法进行优化;列举了包括图形应用程序编程接口(application programming interface,API)和游戏引擎在内的移动端渲染工具。然后,针对Web端在线实时绘制,分析了Web端的3D渲染机制,梳理了以3D场景的轻量化预处理、大规模3D场景的细粒度化网络传输、3D场景的对等传输以及Web3D在线特效渲染为代表的Web端在线绘制的关键技术(尤其面向大规模3D场景),列举了国内外知名Web3D引擎并探讨了主流游戏引擎对Web3D应用的支持。再后,针对云端在线实时绘制,从应用托管、资源调度和串流这三大云平台的核心功能入手,调研了以串流应用优化技术为核心的在线云绘制现状。此后,从多端绘制任务分摊机制入手,分析了以“端云”协同和“端边云”协同为目标的在线多端协同绘制的发展。最后,以当前工业级在线云绘制平台为研究对象,分析了包括微软、英伟达、Unity、酷家乐等一线云绘制企业的在线实时云绘制平台方案,验证了移动在线绘制技术在工业界的实用性。

Mobile internet real-time rendering technology facilitates key technical capabilities on the aspects of three dimensions(3D)visualization,computer vision,virtual reality,augmented reality,extended reality,and meta-universe.3D visualization applications based on online real-time rendering technology have been widely scattered on multiple mobile Internet platforms.3D based Internet applications have been transformed into new online visualization applications with 3D virtual scenes as the main interactive object and intensive immersion to deal with service locations and time constrained issues of the mobile Internet.Our review is focused on four platforms,including mobile terminals,web terminals,cloud terminals and multi-terminals collaboration.First,we investigate the“graphics processing unit”on mobile devices,which is the critical factor of mobile online real-time graphics rendering,and analyze its recent computing ability,designing,and optimization issues.We examined the possibility of hosting desktop-level rendering algorithms on mobile rendering devices,taking the ray-tracing approach as an example.We analyzed the rendering algorithms issues can be accelerated on the power and bandwidth constrained mobile orientation.We reviewed mobile rendering tools like mobile graphics application programming interface(API)and game engines,and conduct comparative category analysis for their performance.Second,we discussed the current Web3D online rendering mechanisms represented by the combination of WebGL2.0 and HTML5 in terms of web-oriented online real-time graphics rendering;we compared the technical features and rendering performance of well-known Web3D engines,and review the support of mainstream game engines for Web3D applications;we examined the challenged technologies of Web3D lightweighting solutions in related to the online real-time rendering of large-scale Web3D scenes like Web3D scene lightweighting and peer-to-peer networking(P2P)progressive transmission.In respect of online real-time graphics cloud rendering,we analyzed the remote rendering architecture of cloud rendering system;We revealed the core service layer of cloud rendering system in terms of cloud services———“platform as a service(PaaS)”,and sorted its three major functions out like application hosting,resource scheduling,and streaming;we analyzed two types of cloudrendering optimization strategies derived from video stream compression and rendering latency mitigation;we described the core application of online cloud rendering system———the“cloud game”,and compared the system framework,technical features,and user experience of some of the latest“cloud game”applications.To enable multi-end collaborative online graphics rendering,we analyzed the rendering-task sharing and collaboration of the end-cloud system;we examined the collaboration pattern and technical features of the end-edge-cloud rendering system based on the introduction of edge computing;we make use of rendering resources toanalyze the load balancing optimization issues originated from computing power deployment and resource scheduling of the multi-end rendering system.Finally,we listed the current industry based online real-time cloud rendering systems,including NVIDIA-derived CloudXR,unity-based render streaming,Microsoft Azure remote rendering,and Kujiale cloud-based home decoration and design platform.We compared these systems through their framework,technical features and rendering effects.