Underwater Inhomogeneous Light Field Based on Improved Convolutional Neural Net Fish Image Recognition

In this paper, artificial intelligence image recognition technology is used to improve the recognition rate of individual domestic fish and reduce the recognition time, aiming at the problem that it is difficult to easily observe the species and growth of domestic fish in the underwater non-uniform light field environment. First, starting from the image data collected by polarizing imaging technology, this paper uses subpixel convolution reconstruction to enhance the image, uses image translation and fill technology to build the family fish database, builds the Adam-Dropout-CNN (A-D-CNN) network model, and its convolution kernel size is 3 × 3. The maximum pooling was used for downsampling, and the discarding operation was added after the full connection layer to avoid the phenomenon of network overfitting. The adaptive motion estimation algorithm was used to solve the gradient sparse problem. The experiment shows that the recognition rate of A-D-CNN is 96.97% when the model is trained under the domestic fish image database, which solves the problem of low recognition rate and slow recognition speed of domestic fish in non-uniform light field.

Integral imaging-based tabletop light field 3D display with large viewing angle

Light field 3D display technology is considered a revolutionary technology to address the critical visual fatigue issues in the existing 3D displays.Tabletop light field 3D display provides a brand-new display form that satisfies multi-user shared viewing and collaborative works,and it is poised to become a potential alternative to the traditional wall and portable display forms.However,a large radial viewing angle and correct radial perspective and parallax are still out of reach for most current tabletop light field 3D displays due to the limited amount of spatial information.To address the viewing angle and perspective issues,a novel integral imaging-based tabletop light field 3D display with a simple flat-panel structure is proposed and developed by applying a compound lens array,two spliced 8K liquid crystal display panels,and a light shaping diffuser screen.The compound lens array is designed to be composed of multiple three-piece compound lens units by employing a reverse design scheme,which greatly extends the radial viewing angle in the case of a limited amount of spatial information and balances other important 3D display parameters.The proposed display has a radial viewing angle of 68.7°in a large display size of 43.5 inches,which is larger than the conventional tabletop light field 3D displays.The radial perspective and parallax are correct,and high-resolution 3D images can be reproduced in large radial viewing positions.We envision that this proposed display opens up possibility for redefining the display forms of consumer electronics.

Single-cell volumetric imaging with light field microscopy: Advances in systems and algorithms

Single-cell volumetric imaging is essential for researching individual characteristics of cells.As a nonscanning imaging technique,lighteld microscopy(LFM)is a critical tool to achieve realtime three-dimensional imaging with the advantage of single-shot.To address the inherent limits including nonuniform resolution and block-wise artifacts,various modied LFM strategies have been developed to provide new insights into the structural and functional information of cells.This review will introduce the principle and development of LFM,discuss the improved approaches based on hardware designs and 3D reconstruction algorithms,and present the applications in single-cell imaging.

LF-UMTI:基于多尺度空角交互的无监督多曝光光场图像融合

光场成像可同时捕获真实场景中光线的强度和方向信息。但受限于成像传感器的势阱容量,现光场相机单曝光捕获的光场图像难以完整记录真实场景中所有的细节信息。为了解决上述问题,本文提出了一种基于多尺度空角交互的无监督多曝光光场成像方法。该方法采用多尺度空角交互策略,以有效提取光场空角特征,同时利用通道维上建模策略以降低计算量来适应光场高维结构。其次,构建了由可逆神经网络导向的光场重建模块,以避免融合伪影并恢复更多细节信息。最后,设计了一种角度一致性损失,其考虑了边界子孔径图像和中心子孔径图像之间的视差变化,以保证融合结果的视差结构。为评估所提方法的性能,建立了一个面向真实场景的多曝光光场基准数据集。实验结果表明,所提方法可在保证角度一致性的前提下重建出具备高对比度和丰富细节的光场图像。与现有方法相比,所提方法在客观质量和主观视觉两方面均取得更好的结果。

使用Light Field Rendering实现森林的实时渲染及光照算法

森林的实时渲染及光照是视景系统中的一个难题.基于图像的渲染方法(IBR)由于渲染速度与模型复杂度无关,被广泛应用于场景重建.基于光流场(Light Field Rendering)的IBR技术,提出一种迭代投射算法来进行外形重建,实现了具有实时光影特征的森林效果.实验表明该算法结合了传统迭代、投射算法各自的优点,在质量和效率方面取得了平衡.

Review of light field technologies

Light fields are vector functions that map the geometry of light rays to the corresponding plenoptic attributes.They describe the holographic information of scenes by representing the amount of light flowing in every direction through every point in space.The physical concept of light fields was first proposed in 1936,and light fields are becoming increasingly important in the field of computer graphics,especially with the fast growth of computing capacity as well as network bandwidth.In this article,light field imaging is reviewed from the following aspects with an emphasis on the achievements of the past five years:(1)depth estimation,(2)content editing,(3)image quality,(4)scene reconstruction and view synthesis,and(5)industrial products because the technologies of lights fields also intersect with industrial applications.State-of-the-art research has focused on light field acquisition,manipulation,and display.In addition,the research has extended from the laboratory to industry.According to these achievements and challenges,in the near future,the applications of light fields could offer more portability,accessibility,compatibility,and ability to visualize the world.

Microlens Light Field Imaging Method Based on Bionic Vision and 3-3 Dimensional Information Transforming

his paper adopts the 3-3-2 information processing method for the capture of moving objects as its premise, and proposes a basic principle of three-dimensional (3D) imaging using biological compound eye. Traditional bionic vision is limited by the available hardware. Therefore, in this paper, the new-generation technology of microlens-array light-field camera is proposed as a potential method for the extraction of depth information from a single image. A significant characteristic of light-field imaging is that it records intensity and directional information from the lights entering the camera. Herein, a refocusing method using light-field image is proposed. By calculating the focusing cost at different depths from the object, the imaging plane of the object is determined, and a depth map is constructed based on the position of the object’s imaging plane. Compared with traditional light-field depth estimation, the depth map calculated by this method can significantly improve resolution and does not depend on the number of light-field microlenses. In addition, considering that software algorithms rely on hardware structure, this study develops an imaging hardware that is only 7 cm long based on the second-generation microlens camera’s structure, further validating its important refocusing characteristics. It thereby provides a technical foundation for 3D imaging with a single camera.

Calibration and data restoration of light field modulated imaging spectrometer

A light field modulated imaging spectrometer （LFMIS） can acquire the spatial-spectral datacube of targets of interest or a scene in a single shot. The spectral information of a point target is imaged on the pixels covered by a microlens. The pixels receive spectral information from different spectral filters to the diffraction and misalignments of the optical components. In this paper, we present a linear spectral multiplexing model of the acquired target spectrum. A calibration method is proposed for calibrating the center wavelengths and bandwidths of channels of an LFMIS system based on the liner-variable filter （LVF） and for determining the spectral multiplexing matrix. In order to improve the accuracy of the restored spectral data, we introduce a reconstruction algorithm based on the total least square （TLS） approach. Simulation and experimental results confirm the performance of the spectrum reconstruction algorithm and validate the feasibility of the proposed calibrating scheme.

Light Field Virtual View Rendering Based on EPI-Representations

Image-Based Rendering （IBR） is one powerful approach for generating virtual views. It can provide convincing animations without an explicit geometric representation. In this paper, several implementations of light field rendering are summa- rized from prior arts. Several characteristics, such as the regu- lar pattern in Epipolar Plane Images （EPIs）, of light field are explored with detail under 1D parallel camera arrangement. It is proved that it is quite efficient to synthesize virtual views for Super Multi-View （SMV） application, which is in the third phase of Free- Viewpoint Television （FTV）. In comparison with convolutional stereo matching method, in which the inter- mediate view is synthesized by the two adjacent views, light field rendering makes use of more views supplied to get the high-quality views.

Light Field Flow Estimation Based on Occlusion Detection

Light field cameras have a wide area of applications, such as digital refocusing, scene depth information extraction and 3-D image reconstruction. By recording the energy and direction information of light field, they can well solve many technical problems that cannot be done by conventional cameras. An important feature of light field cameras is that a microlens array is inserted between the sensor and main lens, through which a series of sub-aperture images of different perspectives are formed. Based on this feature and the full-focus image acquisition technique, we propose a light-field optical flow calculation algorithm, which involves both the depth estimation and the occlusion detection and guarantees the edge-preserving property. This algorithm consists of three steps: 1) Computing the dense optical flow field among a group of sub-aperture images;2) Obtaining a robust depth-estimation by initializing the light-filed optical flow using the linear regression approach and detecting occluded areas using the consistency;3) Computing an improved light-field depth map by using the edge-preserving algorithm to realize interpolation optimization. The reliability and high accuracy of the proposed approach is validated by experimental results.

Experimental Detection of Depth of Field for a Thermal Light Lensless Ghost Imaging System

We propose optical experiments to study the depth of field for a thermal light lensless ghost imaging system. It is proved that the diaphragm is an important factor to influence the depth of field, and the ghost images of two detected objects with longitudinal distance less than the depth of field can be achieved simultaneously. The longitudinal coherence scale of the thermal light lensless ghost imaging determines the depth of field. Theoretical analysis can well explain the experimental results.

一种基于SAM-MSFF网络的低照度目标检测方法

由于低照度图像具有对比度低、细节丢失严重、噪声大等缺点,现有的目标检测算法对低照度图像的检测效果不理想.为此,本文提出一种结合空间感知注意力机制和多尺度特征融合(Spatial-aware Attention Mechanism and Multi-Scale Feature Fusion,SAM-MSFF)的低照度目标检测方法 .该方法首先通过多尺度交互内存金字塔融合多尺度特征,增强低照度图像特征中的有效信息,并设置内存向量存储样本的特征,捕获样本之间的潜在关联性;然后,引入空间感知注意力机制获取特征在空间域的长距离上下文信息和局部信息,从而增强低照度图像中的目标特征,抑制背景信息和噪声的干扰;最后,利用多感受野增强模块扩张特征的感受野,对具有不同感受野的特征进行分组重加权计算,使检测网络根据输入的多尺度信息自适应地调整感受野的大小.在ExDark数据集上进行实验,本文方法的平均精度(mean Average Precision,mAP)达到77.04%,比现有的主流目标检测方法提高2.6%~14.34%.

基于多特征融合和几何感知网络的光场图像编码

为了克服基于单一视差合成的光场图像编码方法在遮挡区域无法恢复纹理细节的问题,提出一种基于多特征融合和几何感知网络的光场图像编码方法,以进一步提升遮挡场景下光场图像的压缩性能.首先,对密集光场稀疏采样,使用通用视频编码器(versatile video coding,VVC)对稀疏光场进行压缩;然后,在解码端使用2个关键分支模块,即视差估计模块和空间角度联合卷积模块,以获取光场图像全局的几何信息,确保在密集纹理和遮挡区域能够更充分地恢复特征;最后,为了挖掘2个分支融合特征的结构信息,构建了双向视图的堆栈结构,并运用几何感知的细化网络以重建高质量的密集光场.实验结果表明,与已有国际上流行的光场图像编码方法相比,所提出的方法具有显著优势.

基于光场成像的多物理场测试技术研究进展

光场成像原理的发展为多物理场测试提供了新的思路与方向。光场相机基于光场成像原理,在主镜头与感光芯片之间增加了微透镜阵列,实现了对入射光线角度信息的捕获。研究人员发展了基于光场成像原理的多物理场测试技术。首先,介绍了光场粒子图像测速技术,使用光场相机拍摄复杂三维流场的粒子图像,随后采用一系列三维体校准、重构和互相关算法获得三维速度场。该技术在保证测量精度的同时实现对三维三分量速度场的捕获,并且能够有效减少相机数量和光学窗口的要求,实现对受限空间内部进行速度场测量。其次,介绍了将光场相机在形貌测量领域中的应用。光场相机通过单个相机、单次拍摄可以获取物体表面的三维点云信息,在极大程度上缩短测量时间的同时,简化成像设备,实现对叶片气膜孔的三维形貌测量。然后,将光场相机应用于温度测量领域,可以简化分光手段,从而完成测温系统的精简,实现单个传感器对光谱强度和角度信息的同时记录。最后,对光场多物理场测试技术进行了总结,并对基于内窥光场成像与超透镜的光场测试技术进行了展望。

基于全局位置迭代PCSS算法的光场PTV气泡跟踪测速方法

光场粒子跟踪测速(PTV)技术能够在单视角条件下重建气液两相流中气泡的三维空间位置,实现气泡运动轨迹的跟踪,为受限空间条件下气泡参数测量提供了解决方案。然而,在气泡浓度较高以及位移量较大的条件下,光场PTV气泡匹配准确率较低,由此导致气泡运动速度场测量结果出现明显的错误矢量。为解决该问题,本文提出了基于全局位置迭代以及极坐标系统相似技术的气泡匹配方法(GPPI-PCSS),通过PCSS匹配方法所获得的气泡三维速度场对单帧图像内所有气泡的位置进行迭代更新,使两帧图像中的气泡位置逐渐重合,由此获得高准确度的气泡匹配结果。通过开展鼓泡床内气泡运动行为光场PTV实验研究,对GPPI-PCSS方法的准确性进行了评价。结果表明:在采样间隔0.5~7ms、空气流量0.15~0.35L/min工况范围内,GPPI-PCSS方法的气泡匹配准确率的平均值为92.65%,分别高于传统PCSS方法的81.38%和松弛方法的84.12%。此外,通过GPPI-PCSS方法所测量得到的气泡最大运动速度范围为38.34~49.87cm/s,与理论计算值一致,由此证明了所提出的GPPI-PCSS方法可以用于光场PTV技术中,在高气泡浓度、大气泡位移条件下获得准确的气泡速度测量结果。

纸膜双层缺陷检测的视觉成像光场设计

为实现包装盒纸质基底层和透明膜层缺陷的同步检测,开展了对纸和膜缺陷的同步成像研究。首先,分别建立了标准球面积分光场、椭球面积分光场和弧面积分光场模型,并利用COMSOL Multiphysics 5.6对3类光场进行射线仿真,对比分析了球面积分光场下射线角度均匀性及辐照均匀性,通过正交仿真优化椭球面积分光场参数;其次,在椭球面积分光场环境、亮场前打光环境、暗场前打光环境下对包装盒成像。与此同时,对包装盒的油污、抵触、开口、泡皱、破损5项常见缺陷依次进行物理检测和机器视觉检测,验证缺陷成像的有效性。试验结果表明,在椭球面积分光场下成像,图像对纸质基底层缺陷特征、透明膜层缺陷特征均有较好的呈现效果,图像上油污、抵触、开口、泡皱、破损的物理检出率分别为96.2%、92.5%、100%、95%、92%,异常检出率分别为98.6%、97.5%、100%、100%、98.4%,缺陷类别检出率分别为97.6%、96%、100%、97%、96%。研究结果表明,椭球面积分光场光路角度和辐照强度均匀,覆透明膜包装盒的缺陷特征呈现清晰,满足工业生产的检测要求。

Efficient and robust CNN-LSTM prediction of flame temperature aided light field online tomography

Light field tomography,an optical combustion diagnostic technology,has recently attracted extensive attention due to its easy implementation and non-intrusion.However,the conventional iterative methods are high data throughput,low efficiency and time-consuming,and the existing machine learning models use the radiation spectrum information of the flame to realize the parameter field measurement at the current time.It is still an offline measurement and cannot realize the online prediction of the instantaneous structure of the actual turbulent combustion field.In this work,a novel online prediction model of flame temperature instantaneous structure based on deep convolutional neural network and long short-term memory(CNN-LSTM)is proposed.The method uses the characteristics of local perception,shared weight,and pooling of CNN to extract the threedimensional(3D)features of flame temperature and outgoing radiation images.Moreover,the LSTM is used to comprehensively utilize the ten historical time series information of high dynamic combustion flame to accurately predict 3D temperature at three future moments.A chaotic time-series dataset based on the flame radiation forward model is built to train and validate the performance of the proposed CNN-LSTM model.It is proven that the CNN-LSTM prediction model can successfully learn the evolution pattern of combustion flame and make accurate predictions.

基于LSMR算法的火焰三维温度场重建

光场成像技术可以同时记录入射光线的空间分布信息和传播方向信息,结合相关反演算法,可以进行火焰三维温度场的重建。最小二乘QR分解算法(least squares via QR factorization,LSQR)可以有效求解基于大型稀疏矩阵的线性问题,但是在对火焰辐射强度求解的过程中,难以保证求解的非负性和准确性。非负最小二乘算法(non-negative least squares,NNLS)可以保证求解的非负性,但是计算效率太低。本文提出将最小二乘残差方法(least square minimal residual,LSMR)用于火焰光场成像三维温度场重建,并研究其重建精度、计算效率、抗噪性能等指标。仿真实验表明,LSMR和NNLS算法可以在不同噪声水平下保证求解火焰辐射强度的非负性。在噪声为5%、10%、15%和20%的情况下,LSMR和NNLS算法对辐射强度的求解精度均比LSQR提高了10%以上,且LSMR算法的求解时间比LSQR和NNLS分别降低了一个数量级和四个数量级。可见,LSMR算法可以在保证求解精度的情况下大幅提高运算效率。最后用LSMR算法对模拟光场火焰进行温度场重建,在不同噪声水平下,平均相对误差都保持在1.2%以内,验证了LSMR算法在重建时的准确性和可靠性。

基于自适应匹配伪序列的光场图像压缩

近年来,光场图像作为一种新的数字图像形式,受到人们极大关注。光场图像具有维度高、信息量大等特点,存储和传输带宽压力巨大,成为其发展的瓶颈,提出一种基于自适应匹配伪序列的光场伪序列构建及压缩方法。首先根据微透镜中心点对原始图像进行重采样得到光场子孔径图像;然后利用光场图像相邻视图间的相关性重新构建伪序列,从而针对不同光场自适应地确定符合多功能视频编码器的光场图像最优编码次序;最后进行视频压缩。实验结果表明,本算法与VTM平台相比具有较好的压缩性能,BD-rate平均降低13.62%,优于其他对比算法。

基于多级残差融合的复杂纹理光场图像深度估计

光场的深度信息可以通过深度学习的光场深度估计算法计算,在图像视差、光场图像边缘以及光场图像的复杂纹理区域,获取高精度深度值仍然具有一定局限性。本文提出了一种用于光场图像深度估计的多级残差融合网络,通过组合残差模块提取多层次的残差特征,在保持网络深度的同时提升了网络对特征的表征能力。利用多级残差融合模块对多层次的残差特征进行融合,以获得包含浅层纹理信息和深层语义信息的融合特征。利用本文方法对HCI4D光场数据集进行处理,图像深度估计的均方误差指标达到1.471,不良像素率指标达到4.208,该实验结果表明本文方法在处理具有复杂遮挡的光场图像区域方面具有良好的处理效果。