Integral imaging-based tabletop light field 3D display with large viewing angle

Light field 3D display technology is considered a revolutionary technology to address the critical visual fatigue issues in the existing 3D displays.Tabletop light field 3D display provides a brand-new display form that satisfies multi-user shared viewing and collaborative works,and it is poised to become a potential alternative to the traditional wall and portable display forms.However,a large radial viewing angle and correct radial perspective and parallax are still out of reach for most current tabletop light field 3D displays due to the limited amount of spatial information.To address the viewing angle and perspective issues,a novel integral imaging-based tabletop light field 3D display with a simple flat-panel structure is proposed and developed by applying a compound lens array,two spliced 8K liquid crystal display panels,and a light shaping diffuser screen.The compound lens array is designed to be composed of multiple three-piece compound lens units by employing a reverse design scheme,which greatly extends the radial viewing angle in the case of a limited amount of spatial information and balances other important 3D display parameters.The proposed display has a radial viewing angle of 68.7°in a large display size of 43.5 inches,which is larger than the conventional tabletop light field 3D displays.The radial perspective and parallax are correct,and high-resolution 3D images can be reproduced in large radial viewing positions.We envision that this proposed display opens up possibility for redefining the display forms of consumer electronics.

A miniaturized spin-exchange relaxation-free atomic magnetometer based on uniform light field

We experimentally study the dynamic characteristics of a miniaturized spin-exchange relaxation-free(SERF) magnetometer based on uniform light field. The ceramic ferrule is used to expand the Gaussian beam to improve light intensity uniformity, while the volume of the sensor is also reduced. This scheme makes the magnetometer have better sensitivity when the detected light intensity is less than 3.16 m W/cm^(2) at 120℃. When the temperature rises to 150℃ the sensitivity under the action of uniform light field is 18.5 f T/Hz^(1/2). The bandwidth of the sensor remains at the original level and meets application needs. The proposed structure improves transverse polarization uniformity within the miniaturized sensor, which is ideal for the magnetoencephalography and magnetocardiography imaging systems.

Single-cell volumetric imaging with light field microscopy: Advances in systems and algorithms

Single-cell volumetric imaging is essential for researching individual characteristics of cells.As a nonscanning imaging technique,lighteld microscopy(LFM)is a critical tool to achieve realtime three-dimensional imaging with the advantage of single-shot.To address the inherent limits including nonuniform resolution and block-wise artifacts,various modied LFM strategies have been developed to provide new insights into the structural and functional information of cells.This review will introduce the principle and development of LFM,discuss the improved approaches based on hardware designs and 3D reconstruction algorithms,and present the applications in single-cell imaging.

Light field microscopy in biological imaging

Light field microscopy(LFM),featured for high threedimensional imaging speed and low pho-totoxicity,has emeged as a technique of choice for instantaneous volumetric inaging.In contrast with other scanning-based three dimensional(3D)imaging approaches,LFM enables to encode 3D spatial information in a snapshot manner,permitting high-speed 3D imaging that is only limited by the frame rate of the camera.In this review,we first introduce the fundamental theory of LFM and current corresponding advanced approaches.Then,we summarize various applica-tions of LFM in biological imaging.

Underwater Inhomogeneous Light Field Based on Improved Convolutional Neural Net Fish Image Recognition

In this paper, artificial intelligence image recognition technology is used to improve the recognition rate of individual domestic fish and reduce the recognition time, aiming at the problem that it is difficult to easily observe the species and growth of domestic fish in the underwater non-uniform light field environment. First, starting from the image data collected by polarizing imaging technology, this paper uses subpixel convolution reconstruction to enhance the image, uses image translation and fill technology to build the family fish database, builds the Adam-Dropout-CNN (A-D-CNN) network model, and its convolution kernel size is 3 × 3. The maximum pooling was used for downsampling, and the discarding operation was added after the full connection layer to avoid the phenomenon of network overfitting. The adaptive motion estimation algorithm was used to solve the gradient sparse problem. The experiment shows that the recognition rate of A-D-CNN is 96.97% when the model is trained under the domestic fish image database, which solves the problem of low recognition rate and slow recognition speed of domestic fish in non-uniform light field.

Backscattering Light Model of Seawater for Modulated Lidar Based on the Stationarity of Light Field

The backscattering signal, which arises from the pulsed laser traveling through water, has limited the lidar system sensitivity and underwater target contrast. The transmitted optical carrier is modulated to be ultrashort pulsed laser and it is effective to suppress the backscattering to adopt the coherent detection technology by identifying the modulation envelope. A nonstationary light field is formed in seawater by the ultrashort pulsed laser. The inherent relationship between the nonstationary light field formed by modulated lidar and the stationary light field formed by conventional lidar was discussed and the backscattering light model of the stationary light field for the ultrashort pulsed laser was proposed. The backscattering signal in modulated lidar system was processed and analyzed in the frequency domain on the basis of the model.

A light field measurement system through PSF estimation by a morphology-based method

Light field imaging technology can obtain three-dimensional(3D)information of a test surface in a single exposure.Traditional light field reconstruction algorithms not only take a long time to trace back to the original image,but also require the exact parameters of the light field system,such as the position and posture of a microlens array(MLA),which will cause errors in the reconstructed image if these parameters cannot be precisely obtained.This paper proposes a reconstruction algorithm for light field imaging based on the point spread function(PSF),which does not require prior knowledge of the system.The accurate PSF derivation process of a light field system is presented,and modeling and simulation were conducted to obtain the relationship between the spatial distribution characteristics and the PSF of the light field system.A morphology-based method is proposed to analyze the overlapping area of the subimages of light field images to identify the accurate spatial location of the MLA used in the system,which is thereafter used to accurately refocus light field imaging.A light field system is built to verify the algorithm’s effectiveness.Experimental results show that the measurement accuracy is increased over 41.0%compared with the traditional method by measuring a step standard.The accuracy of parameters is also improved through a microstructure measurement with a peak-to-valley value of 25.4%and root mean square value of 23.5%improvement.This further validates that the algorithm can effectively improve the refocusing efficiency and the accuracy of the light field imaging results with the superiority of refocusing light field imaging without prior knowledge of the system.The proposed method provides a new solution for fast and accurate 3D measurement based on a light field.

Calibration and data restoration of light field modulated imaging spectrometer

A light field modulated imaging spectrometer （LFMIS） can acquire the spatial-spectral datacube of targets of interest or a scene in a single shot. The spectral information of a point target is imaged on the pixels covered by a microlens. The pixels receive spectral information from different spectral filters to the diffraction and misalignments of the optical components. In this paper, we present a linear spectral multiplexing model of the acquired target spectrum. A calibration method is proposed for calibrating the center wavelengths and bandwidths of channels of an LFMIS system based on the liner-variable filter （LVF） and for determining the spectral multiplexing matrix. In order to improve the accuracy of the restored spectral data, we introduce a reconstruction algorithm based on the total least square （TLS） approach. Simulation and experimental results confirm the performance of the spectrum reconstruction algorithm and validate the feasibility of the proposed calibrating scheme.

New approach for obtaining the time-evolution law of a chaotic light field in a damping gaining-process

A new approach for studying the time-evolution law of a chaotic light field in a damping-gaining coexisting process is presented. The new differential equation for determining the parameter of the density operator p（t） is derived and the solution of f for the damping and gaining processes are studied separately. Our approach is direct and the result is concise since it is not necessary for us to know the Kraus operators in advance.

Time evolution law of a two-mode squeezed light field passing through twin diffusion channels

We explore the time evolution law of a two-mode squeezed light field(pure state)passing through twin diffusion channels,and we find that the final state is a squeezed chaotic light field(mixed state)with entanglement,which shows that even though the two channels are independent of each other,since the two modes of the initial state are entangled with each other,the final state remains entangled.Nevertheless,although the squeezing(entanglement)between the two modes is weakened after the diffusion,it is not completely removed.We also highlight the law of photon number evolution.In the calculation process used in this paper,we make full use of the summation method within the ordered product of operators and the generating function formula for two-variable Hermite polynomials.

Light Field Virtual View Rendering Based on EPI-Representations

Image-Based Rendering （IBR） is one powerful approach for generating virtual views. It can provide convincing animations without an explicit geometric representation. In this paper, several implementations of light field rendering are summa- rized from prior arts. Several characteristics, such as the regu- lar pattern in Epipolar Plane Images （EPIs）, of light field are explored with detail under 1D parallel camera arrangement. It is proved that it is quite efficient to synthesize virtual views for Super Multi-View （SMV） application, which is in the third phase of Free- Viewpoint Television （FTV）. In comparison with convolutional stereo matching method, in which the inter- mediate view is synthesized by the two adjacent views, light field rendering makes use of more views supplied to get the high-quality views.

Review of light field technologies

Light fields are vector functions that map the geometry of light rays to the corresponding plenoptic attributes.They describe the holographic information of scenes by representing the amount of light flowing in every direction through every point in space.The physical concept of light fields was first proposed in 1936,and light fields are becoming increasingly important in the field of computer graphics,especially with the fast growth of computing capacity as well as network bandwidth.In this article,light field imaging is reviewed from the following aspects with an emphasis on the achievements of the past five years:(1)depth estimation,(2)content editing,(3)image quality,(4)scene reconstruction and view synthesis,and(5)industrial products because the technologies of lights fields also intersect with industrial applications.State-of-the-art research has focused on light field acquisition,manipulation,and display.In addition,the research has extended from the laboratory to industry.According to these achievements and challenges,in the near future,the applications of light fields could offer more portability,accessibility,compatibility,and ability to visualize the world.

使用Light Field Rendering实现森林的实时渲染及光照算法

森林的实时渲染及光照是视景系统中的一个难题.基于图像的渲染方法(IBR)由于渲染速度与模型复杂度无关,被广泛应用于场景重建.基于光流场(Light Field Rendering)的IBR技术,提出一种迭代投射算法来进行外形重建,实现了具有实时光影特征的森林效果.实验表明该算法结合了传统迭代、投射算法各自的优点,在质量和效率方面取得了平衡.

Investigation and optimization of sampling characteristics of light field camera for flame temperature measurement

It is essential to investigate the light field camera parameters for the accurate flame temperature measurement because the sampling characteristics of the flame radiation can be varied with them. In this study, novel indices of the light field camera were proposed to investigate the directional and spatial sampling characteristics of the flame radiation. Effects of light field camera parameters such as focal length and magnification of the main lens, focal length and magnification of the microlens were investigated. It was observed that the sampling characteristics of the flame are varied with the different parameters of the light field camera. The optimized parameters of the light field camera were then proposed for the flame radiation sampling. The larger sampling angle(23 times larger) is achieved by the optimized parameters compared to the commercial light field camera parameters. A non-negative least square(NNLS) algorithm was used to reconstruct the flame temperature. The reconstruction accuracy was also evaluated by the optimized parameters. The results suggested that the optimized parameters can provide higher reconstruction accuracy for axisymmetric and non-symmetric flame conditions in comparison to the commercial light field camera.

Light Field Flow Estimation Based on Occlusion Detection

Light field cameras have a wide area of applications, such as digital refocusing, scene depth information extraction and 3-D image reconstruction. By recording the energy and direction information of light field, they can well solve many technical problems that cannot be done by conventional cameras. An important feature of light field cameras is that a microlens array is inserted between the sensor and main lens, through which a series of sub-aperture images of different perspectives are formed. Based on this feature and the full-focus image acquisition technique, we propose a light-field optical flow calculation algorithm, which involves both the depth estimation and the occlusion detection and guarantees the edge-preserving property. This algorithm consists of three steps: 1) Computing the dense optical flow field among a group of sub-aperture images;2) Obtaining a robust depth-estimation by initializing the light-filed optical flow using the linear regression approach and detecting occluded areas using the consistency;3) Computing an improved light-field depth map by using the edge-preserving algorithm to realize interpolation optimization. The reliability and high accuracy of the proposed approach is validated by experimental results.

Efficient and robust CNN-LSTM prediction of flame temperature aided light field online tomography

Light field tomography,an optical combustion diagnostic technology,has recently attracted extensive attention due to its easy implementation and non-intrusion.However,the conventional iterative methods are high data throughput,low efficiency and time-consuming,and the existing machine learning models use the radiation spectrum information of the flame to realize the parameter field measurement at the current time.It is still an offline measurement and cannot realize the online prediction of the instantaneous structure of the actual turbulent combustion field.In this work,a novel online prediction model of flame temperature instantaneous structure based on deep convolutional neural network and long short-term memory(CNN-LSTM)is proposed.The method uses the characteristics of local perception,shared weight,and pooling of CNN to extract the threedimensional(3D)features of flame temperature and outgoing radiation images.Moreover,the LSTM is used to comprehensively utilize the ten historical time series information of high dynamic combustion flame to accurately predict 3D temperature at three future moments.A chaotic time-series dataset based on the flame radiation forward model is built to train and validate the performance of the proposed CNN-LSTM model.It is proven that the CNN-LSTM prediction model can successfully learn the evolution pattern of combustion flame and make accurate predictions.

Detecting the vector of nanoscale light field with atomic defect

Modulation of a vector light field has played an important role in the research of nanophotonics.However,it is still a great challenge to accurately measure the three-dimensional vector distribution at nanoscale.Here,based on the interaction between the light field and atomic-sized nitrogen-vacancy(NV)color center in diamonds,we demonstrate an efficient method for vectorial mapping of the light-field distribution at nanoscale.Single NV centers with different but well-defined symmetry axes are selected and then interact with the same tightly focused light field.The excitation of a single NV center is related to the angle between the NV center axis and the polarization of the light field.Then the fluorescence patterns of different NV centers provide the information on the vectorial light field distribution.Subsequently analyzing the fluorescence patterns with the help of a deep neural network,the intensity and phase of the light-field vectorial components are fully reconstructed with nanometer resolution.The experimental results are in agreement with theoretical calculations.It demonstrates that our method can help to study light–matter interaction at nanoscale and extend the application of vector light fields in research on nanophotonics.

Light field super-resolution using complementary-view feature attention

Light field(LF)cameras record multiple perspectives by a sparse sampling of real scenes,and these perspectives provide complementary information.This information is beneficial to LF super-resolution(LFSR).Compared with traditional single-image super-resolution,LF can exploit parallax structure and perspective correlation among different LF views.Furthermore,the performance of existing methods are limited as they fail to deeply explore the complementary information across LF views.In this paper,we propose a novel network,called the light field complementary-view feature attention network(LF-CFANet),to improve LFSR by dynamically learning the complementary information in LF views.Specifically,we design a residual complementary-view spatial and channel attention module(RCSCAM)to effectively interact with complementary information between complementary views.Moreover,RCSCAM captures the relationships between different channels,and it is able to generate informative features for reconstructing LF images while ignoring redundant information.Then,a maximum-difference information supplementary branch(MDISB)is used to supplement information from the maximum-difference angular positions based on the geometric structure of LF images.This branch also can guide the process of reconstruction.Experimental results on both synthetic and real-world datasets demonstrate the superiority of our method.The proposed LF-CFANet has a more advanced reconstruction performance that displays faithful details with higher SR accuracy than state-of-the-art methods.

Robust Local Light Field Synthesis via Occlusion-aware Sampling and Deep Visual Feature Fusion

Novel view synthesis has attracted tremendous research attention recently for its applications in virtual reality and immersive telepresence.Rendering a locally immersive light field(LF)based on arbitrary large baseline RGB references is a challenging problem that lacks efficient solutions with existing novel view synthesis techniques.In this work,we aim at truthfully rendering local immersive novel views/LF images based on large baseline LF captures and a single RGB image in the target view.To fully explore the precious information from source LF captures,we propose a novel occlusion-aware source sampler(OSS)module which efficiently transfers the pixels of source views to the target view′s frustum in an occlusion-aware manner.An attention-based deep visual fusion module is proposed to fuse the revealed occluded background content with a preliminary LF into a final refined LF.The proposed source sampling and fusion mechanism not only helps to provide information for occluded regions from varying observation angles,but also proves to be able to effectively enhance the visual rendering quality.Experimental results show that our proposed method is able to render high-quality LF images/novel views with sparse RGB references and outperforms state-of-the-art LF rendering and novel view synthesis methods.

Light field information transmission through scattering media with high fidelity

Realizing high-fidelity optical information transmission through a scattering medium is of vital importance in both science and applications,such as short-range fiber communication and optical encryption.Theoretically,an input wavefront can be reconstructed by inverting the transmission matrix of the scattering medium.However,this deterministic method for retrieving light field information encoded in the wavefront has not yet been experimentally demonstrated.Herein,we demonstrate light field information transmission through different scattering media with near-unity fidelity.Multi-dimensional optical information can be delivered through either a multimode fiber or a ground glass without relying on any averaging or approximation,where their Pearson correlation coefficients can be up to 99%.