Review of light field technologies

Light fields are vector functions that map the geometry of light rays to the corresponding plenoptic attributes.They describe the holographic information of scenes by representing the amount of light flowing in every direction through every point in space.The physical concept of light fields was first proposed in 1936,and light fields are becoming increasingly important in the field of computer graphics,especially with the fast growth of computing capacity as well as network bandwidth.In this article,light field imaging is reviewed from the following aspects with an emphasis on the achievements of the past five years:(1)depth estimation,(2)content editing,(3)image quality,(4)scene reconstruction and view synthesis,and(5)industrial products because the technologies of lights fields also intersect with industrial applications.State-of-the-art research has focused on light field acquisition,manipulation,and display.In addition,the research has extended from the laboratory to industry.According to these achievements and challenges,in the near future,the applications of light fields could offer more portability,accessibility,compatibility,and ability to visualize the world.

Calibration and data restoration of light field modulated imaging spectrometer

A light field modulated imaging spectrometer （LFMIS） can acquire the spatial-spectral datacube of targets of interest or a scene in a single shot. The spectral information of a point target is imaged on the pixels covered by a microlens. The pixels receive spectral information from different spectral filters to the diffraction and misalignments of the optical components. In this paper, we present a linear spectral multiplexing model of the acquired target spectrum. A calibration method is proposed for calibrating the center wavelengths and bandwidths of channels of an LFMIS system based on the liner-variable filter （LVF） and for determining the spectral multiplexing matrix. In order to improve the accuracy of the restored spectral data, we introduce a reconstruction algorithm based on the total least square （TLS） approach. Simulation and experimental results confirm the performance of the spectrum reconstruction algorithm and validate the feasibility of the proposed calibrating scheme.

Experimental Detection of Depth of Field for a Thermal Light Lensless Ghost Imaging System

We propose optical experiments to study the depth of field for a thermal light lensless ghost imaging system. It is proved that the diaphragm is an important factor to influence the depth of field, and the ghost images of two detected objects with longitudinal distance less than the depth of field can be achieved simultaneously. The longitudinal coherence scale of the thermal light lensless ghost imaging determines the depth of field. Theoretical analysis can well explain the experimental results.

Efficient and robust CNN-LSTM prediction of flame temperature aided light field online tomography

Light field tomography,an optical combustion diagnostic technology,has recently attracted extensive attention due to its easy implementation and non-intrusion.However,the conventional iterative methods are high data throughput,low efficiency and time-consuming,and the existing machine learning models use the radiation spectrum information of the flame to realize the parameter field measurement at the current time.It is still an offline measurement and cannot realize the online prediction of the instantaneous structure of the actual turbulent combustion field.In this work,a novel online prediction model of flame temperature instantaneous structure based on deep convolutional neural network and long short-term memory(CNN-LSTM)is proposed.The method uses the characteristics of local perception,shared weight,and pooling of CNN to extract the threedimensional(3D)features of flame temperature and outgoing radiation images.Moreover,the LSTM is used to comprehensively utilize the ten historical time series information of high dynamic combustion flame to accurately predict 3D temperature at three future moments.A chaotic time-series dataset based on the flame radiation forward model is built to train and validate the performance of the proposed CNN-LSTM model.It is proven that the CNN-LSTM prediction model can successfully learn the evolution pattern of combustion flame and make accurate predictions.

Robust Local Light Field Synthesis via Occlusion-aware Sampling and Deep Visual Feature Fusion

Novel view synthesis has attracted tremendous research attention recently for its applications in virtual reality and immersive telepresence.Rendering a locally immersive light field(LF)based on arbitrary large baseline RGB references is a challenging problem that lacks efficient solutions with existing novel view synthesis techniques.In this work,we aim at truthfully rendering local immersive novel views/LF images based on large baseline LF captures and a single RGB image in the target view.To fully explore the precious information from source LF captures,we propose a novel occlusion-aware source sampler(OSS)module which efficiently transfers the pixels of source views to the target view′s frustum in an occlusion-aware manner.An attention-based deep visual fusion module is proposed to fuse the revealed occluded background content with a preliminary LF into a final refined LF.The proposed source sampling and fusion mechanism not only helps to provide information for occluded regions from varying observation angles,but also proves to be able to effectively enhance the visual rendering quality.Experimental results show that our proposed method is able to render high-quality LF images/novel views with sparse RGB references and outperforms state-of-the-art LF rendering and novel view synthesis methods.

Light field imaging： models-, calibrations, reconstructions, and applications

Light field imaging is an emerging technology in computational photography areas. Based on innovative designs of the imaging model and the optical path, light field cameras not only record the spatial intensity of three- dimensional （3D） objects, but also capture the angular information of the physical world, which provides new ways to address various problems in computer vision, such as 3D reconstruction, saliency detection, and object recognition. In this paper, three key aspects of light field cameras, i.e., model, calibration, and reconstruction, are reviewed extensively. Furthermore, light field based applications on informatics, physics, medicine, and biology are exhibited. Finally, open issues in light field imaging and long-term application prospects in other natural sciences are discussed.

Deep learning-based scattering removal of light field imaging

Light field imaging has shown significance in research fields for its high-temporal-resolution 3D imaging ability.However,in scenes of light field imaging through scattering,such as biological imaging in vivo and imaging in fog,the quality of 3D reconstruction will be severely reduced due to the scattering of the light field information.In this paper,we propose a deep learning-based method of scattering removal of light field imaging.In this method,a neural network,trained by simulation samples that are generated by light field imaging forward models with and without scattering,is utilized to remove the effect of scattering on light fields captured experimentally.With the deblurred light field and the scattering-free forward model,3D reconstruction with high resolution and high contrast can be realized.We demonstrate the proposed method by using it to realize high-quality 3D reconstruction through a single scattering layer experimentally.

Advances in the light field displays based on integral imaging and holographic techniques(Invited Paper)

Light field displays comprise three-dimensional （3D） visual information presentation devices capable of providing realistic and full parallax autostereoscopic images. In this letter, the recent advances in the light field displays based on integral imaging （II） and holographic techniques are presented. Several advanced approaches to demonstrate the light field displays including viewing angle enhancement techniques of the II display, a fast hologram generation method using graphics processing unit （GPU） and multiple WRPs, and a holographic microscopy to display the living cells are reported. These methods improve some important constraints of the light field displays and add new features.

Light field imaging for computer vision:a survey

Light field(LF)imaging has attracted attention because of its ability to solve computer vision problems.In this paper we briefly review the research progress in computer vision in recent years.For most factors that affect computer vision development,the richness and accuracy of visual information acquisition are decisive.LF imaging technology has made great contributions to computer vision because it uses cameras or microlens arrays to record the position and direction information of light rays,acquiring complete three-dimensional(3D)scene information.LF imaging technology improves the accuracy of depth estimation,image segmentation,blending,fusion,and 3D reconstruction.LF has also been innovatively applied to iris and face recognition,identification of materials and fake pedestrians,acquisition of epipolar plane images,shape recovery,and LF microscopy.Here,we further summarize the existing problems and the development trends of LF imaging in computer vision,including the establishment and evaluation of the LF dataset,applications under high dynamic range(HDR)conditions,LF image enhancement,virtual reality,3D display,and 3D movies,military optical camouflage technology,image recognition at micro-scale,image processing method based on HDR,and the optimal relationship between spatial resolution and four-dimensional(4D)LF information acquisition.LF imaging has achieved great success in various studies.Over the past 25 years,more than 180 publications have reported the capability of LF imaging in solving computer vision problems.We summarize these reports to make it easier for researchers to search the detailed methods for specific solutions.

Digital Refocusing: All-in-Focus Image Rendering Based on Holoscopic 3D Camera

This paper presents an innovative method for digital refocusing of different point in space after capturing and also extracts all-in-focus image. The proposed method extracts all-in-focus image using Michelson contrast formula hence, it helps in calculating the coordinates of the 3D object location. With light field integral camera setup the scene to capture the objects precisely positioned in a measurable distance from the camera therefore, it helps in refocusing process to return the original location where the object is focused;else it will be blurred with less contrast. The highest contrast values at different points in space can return the focused points where the objects are initially positioned as a result;all-in-focus image can also be obtained. Detailed experiments are conducted to demonstrate the credibility of proposed method with results.