元光场事件计算成像(特邀)

Meta Light Field Event Imaging System(Invited)

捕捉动态变化场景的光场信息,在虚拟现实、显微观测、工业检测、天文等重要观测应用中具有重要的应用价值。现有动态光场成像数据通量大,对动态光场数据的采集、传输和处理提出了巨大的挑战。受到昆虫视觉系统的复眼光学结构和快速多目融合感知的浅层视觉神经工作机制启发,提出一种新型动态光场成像范式-元光场事件成像技术来捕获具有高维光场时变属性的元光场事件,可以极大程度增强元光场事件相机对场景的高效光场感知和表征能力。通过充分利用光场结构下事件之间的时空相关性,进一步提出一种类生物视觉的时空事件聚合升维算法,实现光场事件高效且鲁棒地提取。为了证明提出的元光场事件成像技术的应用价值,搭建一套元光场事件成像的原型系统,在不同的计算机视觉任务中广泛地开展真实实验应用,如场景重建、去噪和运动轨迹重建等,结合仿真和真实实验充分证明所提的元光场事件相机的捕获动态光场信息的高效性和鲁棒性。

Capturing the light field data of dynamical scenes has been widely adopted in virtual reality,microscopic observations,industrial inspections,and other important observation applications.The significant amount of data from current light-field imaging systems complicates their acquisition,transmission,and processing.In this study,inspired by the compound-eye optical structure of insects and the visual-neural work mechanism of fast fusion perception,we propose a novel dynamic light-field imaging paradigm for a meta-light-field event imaging system to capture meta-light-field events with high-dimensional light-field time-varying attributes,which can significantly enhance the light-field perception and characterization of a scene.By fully leveraging the spatiotemporal correlation between events under the light-field structure,we further propose a spatiotemporal-correlated event packaging algorithm,which affords efficient and robust extraction of meta-light-field events.To demonstrate the effectiveness of our proposed meta-light-field event imaging system,we construct a prototype system and test various experimental applications in different computer vision tasks,such as scene reconstruction,denoising,and trajectory reconstruction.Both simulation and experiments can fully demonstrate the high efficiency and robustness of our proposed meta-light-field event camera in capturing dynamic light-field information.