3D Depth Measurement for Holoscopic 3D Imaging System

Holoscopic 3D imaging is a true 3D imaging system mimics fly’s eye technique to acquire a true 3D optical model of a real scene. To reconstruct the 3D image computationally, an efficient implementation of an Auto-Feature-Edge (AFE) descriptor algorithm is required that provides an individual feature detector for integration of 3D information to locate objects in the scene. The AFE descriptor plays a key role in simplifying the detection of both edge-based and region-based objects. The detector is based on a Multi-Quantize Adaptive Local Histogram Analysis (MQALHA) algorithm. This is distinctive for each Feature-Edge (FE) block i.e. the large contrast changes (gradients) in FE are easier to localise. The novelty of this work lies in generating a free-noise 3D-Map (3DM) according to a correlation analysis of region contours. This automatically combines the exploitation of the available depth estimation technique with edge-based feature shape recognition technique. The application area consists of two varied domains, which prove the efficiency and robustness of the approach: a) extracting a set of setting feature-edges, for both tracking and mapping process for 3D depthmap estimation, and b) separation and recognition of focus objects in the scene. Experimental results show that the proposed 3DM technique is performed efficiently compared to the state-of-the-art algorithms.

Experimental Study on Full-Surface Buckling of Variable Curvature Cylindrical Shell Using Multi-camera 3D-DIC System

To achieve full-surface strain measurement of variable curvature objects,a 360°3D digital image correlation(DIC)system is proposed.The measurement system consists of four double-camera systems,which capture the object’s entire surface from multiple angles,enabling comprehensive full-surface measurement.To increase the stitching quality,a hierarchical coordinate matching method is proposed.Initially,a 3D rigid body calibration auxiliary block is employed to track motion trajectory,which enables preliminary matching of four 3D-DIC sub-systems.Subsequently,secondary precise matching is performed based on feature points on the test specimen’s surface.Through the hierarchical coordinate matching method,the local 3D coordinate systems of each double-camera system are unified into a global coordinate system,achieving 3D surface reconstruction of the variable curvature cylindrical shell,and error analysis is conducted on the results.Furthermore,axial compression buckling experiment is conducted to measure the displacement and strain fields on the cylindrical shell’s surface.The experimental results are compared with the finite element analysis,validating the accuracy and effectiveness of the proposed multi-camera 3D-DIC measuring system.

Structured-Light-Field 3D Imaging System With Coaxial Projection

This paper presents a novel 3D measurement method for a light field camera(LFC)in which 3D information of object space is encoded by a microlens array(MLA).The light ray corresponding to each pixel of the LFC is calibrated.Once the matching points from at least two subviews exhibit sub-pixel accuracy,the 3D coordinates can be calculated optimally by intersecting light rays of these points matched through phase coding.Moreover,the proposed method obtains high-resolved results that exceed the subview resolution due to the virtual continuous phase search strategy.Finally,we combine the LFC and coaxial projection to solve the 3D data loss caused by shadowing and occlusion problems.Experimental results verify the feasibility of the proposed method,and the measurement error is about 30μm in a depth range of 60 mm.

3D imaging lipidometry in single cell by in-flow holographic tomography

The most recent discoveries in the biochemical field are highlighting the increasingly important role of lipid droplets(LDs)in several regulatory mechanisms in living cells.LDs are dynamic organelles and therefore their complete characterization in terms of number,size,spatial positioning and relative distribution in the cell volume can shed light on the roles played by LDs.Until now,fluorescence microscopy and transmission electron microscopy are assessed as the gold standard methods for identifying LDs due to their high sensitivity and specificity.However,such methods generally only provide 2D assays and partial measurements.Furthermore,both can be destructive and with low productivity,thus limiting analysis of large cell numbers in a sample.Here we demonstrate for the first time the capability of 3D visualization and the full LD characterization in high-throughput with a tomographic phase-contrast flow-cytometer,by using ovarian cancer cells and monocyte cell lines as models.A strategy for retrieving significant parameters on spatial correlations and LD 3D positioning inside each cell volume is reported.The information gathered by this new method could allow more in depth understanding and lead to new discoveries on how LDs are correlated to cellular functions.

Airborne sparse flight array SAR 3D imaging based on compressed sensing in frequency domain

In airborne array synthetic aperture radar(SAR), the three-dimensional(3D) imaging performance and cross-track resolution depends on the length of the equivalent array. In this paper, Barker sequence criterion is used for sparse flight sampling of airborne array SAR, in order to obtain high cross-track resolution in as few times of flights as possible. Under each flight, the imaging algorithm of back projection(BP) and the data extraction method based on modified uniformly redundant arrays(MURAs) are utilized to obtain complex 3D image pairs. To solve the side-lobe noise in images, the interferometry between each image pair is implemented, and compressed sensing(CS) reconstruction is adopted in the frequency domain. Furthermore, to restore the geometrical relationship between each flight, the phase information corresponding to negative MURA is compensated on each single-pass image reconstructed by CS. Finally,by coherent accumulation of each complex image, the high resolution in cross-track direction is obtained. Simulations and experiments in X-band verify the availability.

Monopulse instantaneous 3D imaging for wideband radar system

To avoid the complicated motion compensation in interferometric inverse synthetic aperture(InISAR)and achieve realtime three-dimensional(3 D)imaging,a novel approach for 3 D imaging of the target only using a single echo is presented.This method is based on an isolated scatterer model assumption,thus the scatterers in the beam can be extracted individually.The radial range of each scatterer is estimated by the maximal likelihood estimation.Then,the horizontal and vertical wave path difference is derived by using the phase comparison technology for each scatterer,respectively.Finally,by utilizing the relationship among the 3 D coordinates,the radial range,the horizontal and vertical wave path difference,the 3 D image of the target can be reconstructed.The reconstructed image is free from the limitation in InISAR that the image plane depends on the target's own motions and on its relative position with respect to the radar.Furthermore,a phase ambiguity resolution method is adopted to ensure the success of the 3 D imaging when phase ambiguity occurs.It can be noted that the proposed phase ambiguity resolution method only uses one antenna pair and does not require a priori knowledge,whereas the existing phase ambiguity methods may require two or more antenna pairs or a priori knowledge for phase unwarping.To evaluate the performance of the proposed method,the theoretical analyses on estimation accuracy are presented and the simulations in various scenarios are also carried out.

Reduced Imaging Time and Improved Image Quality of 3D Isotropic T2-Weighted Magnetic Resonance Imaging with Compressed Sensing for the Female Pelvis

This study is to compare three-dimensional(3D)isotropic T2-weighted magnetic resonance imaging(MRI)with compressed sensing-sampling perfection with application optimized contrast(CS-SPACE)and the conventional image(3D-SPACE)sequence in terms of image quality,estimated signal-to-noise ratio(SNR),relative contrast-to-noise ratio(CNR),and the lesions’conspicuous of the female pelvis.Thirty-six females(age:51,28-73)with cervical carcinoma(n=20),rectal carcinoma(n=7),or uterine fibroid(n=9)were included.Patients underwent magnetic resonance(MR)imaging at a 3T scanner with the sequences of 3D-SPACE,CS-SPACE,and twodimensional(2D)T2-weighted turbo-spin echo(TSE).Quantitative analyses of estimated SNR and relative CNR between tumors and other tissues,image quality,and tissue conspicuity were performed.Two radiologists assessed the difference in diagnostic findings for carcinoma.Quantitative values and qualitative scores were analyzed,respectively.The estimated SNR and the relative CNR of tumor-to-muscle obturator internus,tumor-to-myometrium,and myometrium-to-muscle obturator internus was comparable between 3D-SPACE and CS-SPACE.The overall image quality and the conspicuity of the lesion scores of the CS-SPACE were higher than that of the 3D-SPACE(P<0.01).The CS-SPACE sequence offers shorter scan time,fewer artifacts,and comparable SNR and CNR to conventional 3D-SPACE,and has the potential to improve the performance of T2-weighted images.

Development and prospect of acoustic reflection imaging logging processing and interpretation method

Acoustic reflection imaging logging technology can detect and evaluate the development of reflection anomalies,such as fractures,caves and faults,within a range of tens of meters from the wellbore,greatly expanding the application scope of well logging technology.This article reviews the development history of the technology and focuses on introducing key methods,software,and on-site applications of acoustic reflection imaging logging technology.Based on the analyses of major challenges faced by existing technologies,and in conjunction with the practical production requirements of oilfields,the further development directions of acoustic reflection imaging logging are proposed.Following the current approach that utilizes the reflection coefficients,derived from the computation of acoustic slowness and density,to perform seismic inversion constrained by well logging,the next frontier is to directly establish the forward and inverse relationships between the downhole measured reflection waves and the surface seismic reflection waves.It is essential to advance research in imaging of fractures within shale reservoirs,the assessment of hydraulic fracturing effectiveness,the study of geosteering while drilling,and the innovation in instruments of acoustic reflection imaging logging technology.

Stimulated Raman scattering microscopy with phase-controlled light focusing and aberration correction for rapid and label-free, volumetric deep tissue imaging

We report a novel stimulated Raman scattering(SRS)microscopy technique featuring phase-controlled light focusing and aberration corrections for rapid,deep tissue 3D chemical imaging with subcellular resolution.To accomplish phasecontrolled SRS(PC-SRS),we utilize a single spatial light modulator to electronically tune the axial positioning of both the shortened-length Bessel pump and the focused Gaussian Stokes beams,enabling z-scanning-free optical sectioning in the sample.By incorporating Zernike polynomials into the phase patterns,we simultaneously correct the system aberrations at two separate wavelengths(~240 nm difference),achieving a~3-fold enhancement in signal-to-noise ratio over the uncorrected imaging system.PC-SRS provides>2-fold improvement in imaging depth in various samples(e.g.,polystyrene bead phantoms,porcine brain tissue)as well as achieves SRS 3D imaging speed of~13 Hz per volume for real-time monitoring of Brownian motion of polymer beads in water,superior to conventional point-scanning SRS 3D imaging.We further utilize PC-SRS to observe the metabolic activities of the entire tumor liver in living zebrafish in cellsilent region,unraveling the upregulated metabolism in liver tumor compared to normal liver.This work shows that PCSRS provides unprecedented insights into morpho-chemistry,metabolic and dynamic functioning of live cells and tissue in real-time at the subcellular level.

The Application of Digital Optical 3D Image Analyzer Evaskin in The Evaluation of Wrinkles

To verify the effectiveness of digital optical 3D image analyzer EvaSKIN in the objective and quantitative evaluation of wrinkles.A total of 115 subjects were recruited,the facial images of the subjects were collected by digital optical 3D image analyzer and manual camera,the changes of crow’s feet with age were analyzed.Pictures obtained by manual photography can be directly used for observation and preliminary grading of wrinkles.However,the requirements for evaluators are high,and the results are prone to errors,which will affect the accuracy of the evaluation.Therefore,skilled raters are needed.Compared with the manual photography method,the digital optical 3D image analyzer EvaSKIN can realize three-dimensional extraction of wrinkles,and obtain the change trend of crow’s feet with age.20~30 years old,wrinkles begin to appear slowly;wrinkles will increase rapidly at the age of 30~50;The length of 50~60 year old wrinkles is basically fixed,the wrinkles develop longitudewise,gradually widen and deepen,and the area,depth and volume increase is obvious,and the skin aging condition is intensified.the digital optical 3D image analyzer EvaSKIN realizes the 3D extraction of wrinkles,quantifies the circumference,area,average depth,maximum depth and volume of wrinkles,realizes the objective and quantitative evaluation of wrinkle state,is more accurate in the measurement of wrinkles,and provides a new instrument and method for the evaluation of wrinkles.it is a perfect and supplement to the traditional evaluation methods,and to a certain extent,it helps the research and development and evaluation institutions of cosmetics to obtain more abundant and three-dimensional data support.

Near-field 3D imaging approach combining MJSR and FGG-NUFFT

A near-field three-dimensional(3 D)imaging method combining multichannel joint sparse recovery(MJSR)and fast Gaussian gridding nonuniform fast Fourier transform(FGGNUFFT)is proposed,based on a perfect combination of the compressed sensing(CS)theory and the matched filtering(MF)technique.The approach has the advantages of high precision and high efficiency:multichannel joint sparse constraint is adopted to improve the problem that the images recovered by the single channel imaging algorithms do not necessarily share the same positions of the scattering centers;the CS dictionary is constructed by combining MF and FGG-NUFFT,so as to improve the imaging efficiency and memory requirement.Firstly,a near-field 3 D imaging model of joint sparse recovery is constructed by combining the MF-based imaging method.Secondly,FGG-NUFFT and reverse FGG-NUFFT are used to replace the interpolation and Fourier transform in MF-based imaging methods,and a sensing matrix with high precision and high efficiency is constructed according to the traditional imaging process.Thirdly,a fast imaging recovery is performed by using the improved separable surrogate functionals(SSF)optimization algorithm,only with matrix and vector multiplication.Finally,a 3 D imagery of the near-field target is obtained by using both the horizontal and the pitching interferometric phase information.This paper contains two imaging models,the only difference is the sub-aperture method used in inverse synthetic aperture radar(ISAR)imaging.Compared to traditional CS-based imaging methods,the proposed method includes both forward transform and inverse transform in each iteration,which improves the quality of reconstruction.The experimental results show that,the proposed method improves the imaging accuracy by about O(10),accelerates the imaging speed by five times and reduces the memory usage by about O(10~2).

Deep learning enhanced NIR-Ⅱ volumetric imaging of whole mice vasculature

Fluorescence imaging through the second near-infrared window(NIR-Ⅱ,1000–1700 nm) allows in-depth imaging.However, current imaging systems use wide-field illumination and can only provide low-contrast 2D information, without depth resolution. Here, we systematically apply a light-sheet illumination, a time-gated detection, and a deep-learning algorithm to yield high-contrast high-resolution volumetric images. To achieve a large Fo V(field of view) and minimize the scattering effect, we generate a light sheet as thin as 100.5 μm with a Rayleigh length of 8 mm to yield an axial resolution of 220 μm. To further suppress the background, we time-gate to only detect long lifetime luminescence achieving a high contrast of up to 0.45 Icontrast. To enhance the resolution, we develop an algorithm based on profile protrusions detection and a deep neural network and distinguish vasculature from a low-contrast area of 0.07 Icontrast to resolve the 100μm small vessels. The system can rapidly scan a volume of view of 75 × 55 × 20 mm3and collect 750 images within 6mins. By adding a scattering-based modality to acquire the 3D surface profile of the mice skin, we reveal the whole volumetric vasculature network with clear depth resolution within more than 1 mm from the skin. High-contrast large-scale 3D animal imaging helps us expand a new dimension in NIR-Ⅱ imaging.

The diagnostic value of 3D spiral CT imaging of cholangiopancreatic ducts on obstructive jaundice

Objective: Computerized tomography (CT) plays an important role in the diagnosis of diseases of biliary tract. Recently, three dimensions (3D) spiral CT imaging has been used in surgical diseases gradually. This study was designed to evaluate the diagnostic value of 3D spiral CT imaging of cholangiopancreatic ducts on obstructive jaundice. Methods: Thirty patients with obstructive jaundice had received B-mode ultrasonography, CT, percutaneous transhepatic cholangiography (PTC) or endoscopic retrograde cholangiopancreatography (ERCP), and 3D spiral CT imaging of cholangiopancreatic ducts preoperatively. Then the diagnose accordance rate of these examinational methods were compared after operations. Results: The diagnose accordance rate of 3D spiral CT imaging of cholangiopancreatic ducts was higher than those of B-mode ultrasonography, CT, or single PTC or ERCP, which showed clear images of bile duct tree and pathological changes. As to malignant obstructive jaundice, this examinational technique could clearly display the adjacent relationship between tumor and liver tissue, biliary ducts, blood vessels, and intrahepatic metastases. Conclusion: 3D spiral CT imaging of cholangiopancreatic ducts has significant value for obstructive diseases of biliary ducts, which provides effective evidence for the feasibility of tumor-resection and surgical options.

Short‐term and long‐term memory self‐attention network for segmentation of tumours in 3D medical images

Tumour segmentation in medical images(especially 3D tumour segmentation)is highly challenging due to the possible similarity between tumours and adjacent tissues,occurrence of multiple tumours and variable tumour shapes and sizes.The popular deep learning‐based segmentation algorithms generally rely on the convolutional neural network(CNN)and Transformer.The former cannot extract the global image features effectively while the latter lacks the inductive bias and involves the complicated computation for 3D volume data.The existing hybrid CNN‐Transformer network can only provide the limited performance improvement or even poorer segmentation performance than the pure CNN.To address these issues,a short‐term and long‐term memory self‐attention network is proposed.Firstly,a distinctive self‐attention block uses the Transformer to explore the correlation among the region features at different levels extracted by the CNN.Then,the memory structure filters and combines the above information to exclude the similar regions and detect the multiple tumours.Finally,the multi‐layer reconstruction blocks will predict the tumour boundaries.Experimental results demonstrate that our method outperforms other methods in terms of subjective visual and quantitative evaluation.Compared with the most competitive method,the proposed method provides Dice(82.4%vs.76.6%)and Hausdorff distance 95%(HD95)(10.66 vs.11.54 mm)on the KiTS19 as well as Dice(80.2%vs.78.4%)and HD95(9.632 vs.12.17 mm)on the LiTS.

3D Confocal Raman Imaging of Oil-Rich Emulsion from Enzyme-Assisted Aqueous Extraction of Extruded Soybean Powder

The understanding of the structure morphology of oil-rich emulsion from enzyme-assisted extraction processing(EAEP)was a critical step to break the oil-rich emulsion structure in order to recover oil.Albeit EAEP method has been applied as an alternative way to conventional solvent extraction method,the structure morphology of oil-rich emulsion was still unclear.The current study aimed to investigate the structure morphology of oil-rich emulsion from EAEP using 3 D confocal Raman imaging technique.With increasing the enzymatic hydrolysis duration from 1 to 3 h,the stability of oil-rich emulsion was decreased as visualized in the 3 D confocal Raman images that the protein and oil were mixed together.The subsequent Raman spectrum analysis further revealed that the decreased stability of oil-rich emulsion was due to the protein aggregations via SS bonds or protein-lipid interactions.The conformational transfer in protein indicated the formation of a compact structure.

A modified OMP method for multi-orbit three dimensional ISAR imaging of the space target

The conventional two dimensional(2D)inverse synthetic aperture radar(ISAR)imaging fails to provide the targets'three dimensional(3D)information.In this paper,a 3D ISAR imaging method for the space target is proposed based on mutliorbit observation data and an improved orthogonal matching pursuit(OMP)algorithm.Firstly,the 3D scattered field data is converted into a set of 2D matrix by stacking slices of the 3D data along the elevation direction dimension.Then,an improved OMP algorithm is applied to recover the space target's amplitude information via the 2D matrix data.Finally,scattering centers can be reconstructed with specific three dimensional locations.Numerical simulations are provided to demonstrate the effectiveness and superiority of the proposed 3D imaging method.

A network lightweighting method for difficult segmentation of 3D medical images

Currently,deep learning is widely used in medical image segmentation and has achieved good results.However,3D medical image segmentation tasks with diverse lesion characters,blurred edges,and unstable positions require complex networks with a large number of parameters.It is computationally expensive and results in high requirements on equipment,making it hard to deploy the network in hospitals.In this work,we propose a method for network lightweighting and applied it to a 3D CNN based network.We experimented on a COVID-19 lesion segmentation dataset.Specifically,we use three cascaded one-dimensional convolutions to replace a 3D convolution,and integrate instance normalization with the previous layer of one-dimensional convolutions to accelerate network inference.In addition,we simplify test-time augmentation and deep supervision of the network.Experiments show that the lightweight network can reduce the prediction time of each sample and the memory usage by 50%and reduce the number of parameters by 60%compared with the original network.The training time of one epoch is also reduced by 50%with the segmentation accuracy dropped within the acceptable range.

A 128×128 SPAD LiDAR sensor with column-parallel 25 ps resolution TA-ADCs

This paper presents a design of single photon avalanche diode(SPAD)light detection and ranging(LiDAR)sensor with 128×128 pixels and 128 column-parallel time-to-analog-merged-analog-to-digital converts(TA-ADCs).Unlike the conventional TAC-based SPAD LiDAR sensor,in which the TAC and ADC are separately implemented,we propose to merge the TAC and ADC by sharing their capacitors,thus avoiding the analog readout noise of TAC’s output buffer,improving the conversion rate,and reducing chip area.The reverse start-stop logic is employed to reduce the power of the TA-ADC.Fabricated in a 180 nm CMOS process,our prototype sensor exhibits a timing resolution of 25 ps,a DNL of+0.30/−0.77 LSB,an INL of+1.41/−2.20 LSB,and a total power consumption of 190 mW.A flash LiDAR system based on this sensor demonstrates the function of 2D/3D imaging with 128×128 resolution,25 kHz inter-frame rate,and sub-centimeter ranging precision.

SSL Depth: self-supervised learning enables 16× speedup in confocal microscopy-based 3D surface imaging [Invited]

In scientific and industrial research, three-dimensional (3D) imaging, or depth measurement, is a critical tool that provides detailed insight into surface properties. Confocal microscopy, known for its precision in surface measurements, plays a key role in this field. However, 3D imaging based on confocal microscopy is often challenged by significant data requirements and slow measurement speeds. In this paper, we present a novel self-supervised learning algorithm called SSL Depth that overcomes these challenges. Specifically, our method exploits the feature learning capabilities of neural networks while avoiding the need for labeled data sets typically associated with supervised learning approaches. Through practical demonstrations on a commercially available confocal microscope, we find that our method not only maintains higher quality, but also significantly reduces the frequency of the z-axis sampling required for 3D imaging. This reduction results in a remarkable 16×measurement speed, with the potential for further acceleration in the future. Our methodological advance enables highly efficient and accurate 3D surface reconstructions, thereby expanding the potential applications of confocal microscopy in various scientific and industrial fields.

Dynamic 3D shape reconstruction under complex reflection and transmission conditions using multi-scale parallel single-pixel imaging

Depth measurement and three-dimensional(3D)imaging under complex reflection and transmission conditions are challenging and even impossible for traditional structured light techniques,owing to the precondition of point-to-point triangulation.Despite recent progress in addressing this problem,there is still no efficient and general solution.Herein,a Fourier dual-slice projection with depth-constrained localization is presented to separate and utilize different illumination and reflection components efficiently,which can significantly decrease the number of projection patterns in each sequence from thousands to fifteen.Subsequently,multi-scale parallel single-pixel imaging(MS-PSI)is proposed based on the established and proven position-invariant theorem,which breaks the local regional assumption and enables dynamic 3D reconstruction.Our methodology successfully unveils unseen-before capabilities such as(1)accurate depth measurement under interreflection and subsurface scattering conditions,(2)dynamic measurement of the time-varying high-dynamic-range scene and through thin volumetric scattering media at a rate of 333 frames per second;(3)two-layer 3D imaging of the semitransparent surface and the object hidden behind it.The experimental results confirm that the proposed method paves the way for dynamic 3D reconstruction under complex optical field reflection and transmission conditions,benefiting imaging and sensing applications in advanced manufacturing,autonomous driving,and biomedical imaging.