分段式多层卷积神经网络渲染图像降噪模型

Sectional Multi-Layer Convolutional Neural Network Based Real-Time Denoiser for Low Sampling Rate Rendering

全局光照渲染技术在虚拟现实应用中日益普及,但其图像高分辨率采样带来的高时间成本严重影响用户感受.为解决上述问题,提出分段式卷积神经网络模型,对低分辨率采样渲染结果进行实时降噪并获得更高质量的渲染图像结果.该模型分为2段,针对已有降噪模型处理时序渲染结果序列时出现的不稳定性瓶颈,前段使用多层跳跃连接的循环卷积神经网络将渲染结果以序列为单位进行处理,保障降噪结果的时序稳定性;针对降噪模型在时序降噪中的效果瑕疵,后段串联多层渲染图像降噪卷积神经网络对处理结果进行优化;为加快模型训练速度并进一步提升降噪效果,使用低分辨率采样的场景反射率图、法线向量图、场景深度图、阴影图等渲染辅助图像信息作为辅助输入.所提模型综合了已有图像和视频降噪模型的优点,在5种自定义场景上的降噪实验结果表明,该模型具有良好的时序稳定性和降噪效果,镜面处噪点数量明显少于当前主流的OptiX降噪器;在降噪结果与目标图像的结构相似性(SSIM)指标上,与OptiX降噪器相比,该模型在5个场景中分别有5.8%,12.2%,1.5%,4.7%和1.8%的提升.

The application scenarios of global illumination rendering have become more and more extensive.However,due to the limited computing capability of computer hardware,high samples-per-pixel(SPP)rendering will lead to terrible user experience.We offered a solution using the deep learning method to restore low SPP pictures through sectional convolutional neural networks(CNNs)and auxiliary inputs.Our model achieved better restoration effects and real-time denoising under the premise of reducing sampling cost.Specifically,we used a structure with multiple skip connections based on recurrent convolutional neural network(RCNN)to handle the input picture sequence,which overcame the shortage of picture denoiser in terms of time stability.To deal with the limited reconstruction ability of RCNN,we used multiple denoising convolutional neural network(DnCNN)layers to further process the output data.In order to obtain a better real-time denoising effect,we introduced auxiliary inputs to our model,including albedo maps,normal vector maps,depth maps and shadowing maps gathered from the low SPP rendering process.Our model synthesized the virtue of picture denoiser and video denoiser.On the 5 different customized scenes we made,our model shows gratifying time stability and outperforms the widely used OptiX denoiser.Using the structural similarity(SSIM)as objective indicator,our model shows 5.8%,12.2%,1.5%,4.7%,1.8%improvement on the 5 different scenes respectively when compared with OptiX denoiser.