基于时空全局优化的视频修复

A Global Space-Time Optimization Framework for Video Completion

提出一种基于样本的视频修复算法,将视频修复问题看作是嵌入到时空视频体中的一个三维图的离散的全局优化问题,可有效地修复视频中较大破损的区域.首先建立待修复视频的马尔科夫随机场(MRF),然后设置新的目标函数,将修复问题转化为马尔科夫随机场的多标号问题,使待修复区域与其周围区域在颜色的相似性和运动的相似性保持一致性.进而提出了灵巧置信传播求解算法,有效求解此目标函数,大大降低了标准置信传播算法的时空复杂度.还提出了与视觉相关的像素权值的概念,使得算法能更好地按照视觉合理性计算时空块的相似度.对复杂动态场修复结果实验表明,较现有算法,文中算法能更好地修复出显著结构和运动信息.

This paper presents a new exemplar-based framework for completing video with large spaee-time holes, eonsidering video completion as a proeess of global optimization on a 3D graph embedded in the spaee-time video volume. It first sets up a 3D Markov Random Field （MRF） on the missing holes, then proposes a new global diserete optimization funetion, whieh eonverts the video eompletion into multi-label problem in MRF, and enforees global spatio temporal eonsisteney among patehes that fill the holes, in terms of both eolor similarity and motion similarity. The optimization is aeeomplished by a novel algorithm, ealled smart belief propagation, whieh is signifieantly better than the standard belief propagation, in terms of eompletion quality, eomputational effieieney and storage size. In addition, by ineorporating motion information, we introduee a more effeetive and aeeurate spatio-temporal pateh similarity metrie, whieh leads to more visually pleasing eompletion results. Experiments on several complex dynamic scenes demonstrate that the missing salient video structures and motion information are much better restored by the proposed methods than the existing ones.