Estimation-free spatial-domain image reconstruction of structured illumination microscopy

Structured illumination microscopy(SIM)achieves super-resolution(SR)by modulating the high-frequency information of the sample into the passband of the optical system and subsequent image reconstruction.The traditional Wiener-filtering-based reconstruction algorithm operates in the Fourier domain,it requires prior knowledge of the sinusoidal illumination patterns which makes the time-consuming procedure of parameter estimation to raw datasets necessary,besides,the parameter estimation is sensitive to noise or aberration-induced pattern distortion which leads to reconstruction artifacts.Here,we propose a spatial-domain image reconstruction method that does not require parameter estimation but calculates patterns from raw datasets,and a reconstructed image can be obtained just by calculating the spatial covariance of differential calculated patterns and differential filtered datasets(the notch filtering operation is performed to the raw datasets for attenuating and compensating the optical transfer function(OTF)).Experiments on reconstructing raw datasets including nonbiological,biological,and simulated samples demonstrate that our method has SR capability,high reconstruction speed,and high robustness to aberration and noise.

Pyramid Separable Channel Attention Network for Single Image Super-Resolution

Single Image Super-Resolution(SISR)technology aims to reconstruct a clear,high-resolution image with more information from an input low-resolution image that is blurry and contains less information.This technology has significant research value and is widely used in fields such as medical imaging,satellite image processing,and security surveillance.Despite significant progress in existing research,challenges remain in reconstructing clear and complex texture details,with issues such as edge blurring and artifacts still present.The visual perception effect still needs further enhancement.Therefore,this study proposes a Pyramid Separable Channel Attention Network(PSCAN)for the SISR task.Thismethod designs a convolutional backbone network composed of Pyramid Separable Channel Attention blocks to effectively extract and fuse multi-scale features.This expands the model’s receptive field,reduces resolution loss,and enhances the model’s ability to reconstruct texture details.Additionally,an innovative artifact loss function is designed to better distinguish between artifacts and real edge details,reducing artifacts in the reconstructed images.We conducted comprehensive ablation and comparative experiments on the Arabidopsis root image dataset and several public datasets.The experimental results show that the proposed PSCAN method achieves the best-known performance in both subjective visual effects and objective evaluation metrics,with improvements of 0.84 in Peak Signal-to-Noise Ratio(PSNR)and 0.017 in Structural Similarity Index(SSIM).This demonstrates that the method can effectively preserve high-frequency texture details,reduce artifacts,and have good generalization performance.

Multi-prior physics-enhanced neural network enables pixel super-resolution and twin-imagefree phase retrieval from single-shot hologram

Digital in-line holographic microscopy(DIHM)is a widely used interference technique for real-time reconstruction of living cells’morphological information with large space-bandwidth product and compact setup.However,the need for a larger pixel size of detector to improve imaging photosensitivity,field-of-view,and signal-to-noise ratio often leads to the loss of sub-pixel information and limited pixel resolution.Additionally,the twin-image appearing in the reconstruction severely degrades the quality of the reconstructed image.The deep learning(DL)approach has emerged as a powerful tool for phase retrieval in DIHM,effectively addressing these challenges.However,most DL-based strategies are datadriven or end-to-end net approaches,suffering from excessive data dependency and limited generalization ability.Herein,a novel multi-prior physics-enhanced neural network with pixel super-resolution(MPPN-PSR)for phase retrieval of DIHM is proposed.It encapsulates the physical model prior,sparsity prior and deep image prior in an untrained deep neural network.The effectiveness and feasibility of MPPN-PSR are demonstrated by comparing it with other traditional and learning-based phase retrieval methods.With the capabilities of pixel super-resolution,twin-image elimination and high-throughput jointly from a single-shot intensity measurement,the proposed DIHM approach is expected to be widely adopted in biomedical workflow and industrial measurement.

Triple-path feature transform network for ring-array photoacoustic tomography image reconstruction

Photoacoustic imaging(PAI)is a noninvasive emerging imaging method based on the photoacoustic effect,which provides necessary assistance for medical diagnosis.It has the characteristics of large imaging depth and high contrast.However,limited by the equipment cost and reconstruction time requirements,the existing PAI systems distributed with annular array transducers are difficult to take into account both the image quality and the imaging speed.In this paper,a triple-path feature transform network(TFT-Net)for ring-array photoacoustic tomography is proposed to enhance the imaging quality from limited-view and sparse measurement data.Specifically,the network combines the raw photoacoustic pressure signals and conventional linear reconstruction images as input data,and takes the photoacoustic physical model as a prior information to guide the reconstruction process.In addition,to enhance the ability of extracting signal features,the residual block and squeeze and excitation block are introduced into the TFT-Net.For further efficient reconstruction,the final output of photoacoustic signals uses‘filter-then-upsample’operation with a pixel-shuffle multiplexer and a max out module.Experiment results on simulated and in-vivo data demonstrate that the constructed TFT-Net can restore the target boundary clearly,reduce background noise,and realize fast and high-quality photoacoustic image reconstruction of limited view with sparse sampling.

Model-driven CT reconstruction algorithm for nano-resolution x-ray phase contrast imaging

The low-density imaging performance of a zone plate-based nano-resolution hard x-ray computed tomography(CT)system can be significantly improved by incorporating a grating-based Lau interferometer. Due to the diffraction, however,the acquired nano-resolution phase signal may suffer splitting problem, which impedes the direct reconstruction of phase contrast CT(nPCT) images. To overcome, a new model-driven nPCT image reconstruction algorithm is developed in this study. In it, the diffraction procedure is mathematically modeled into a matrix B, from which the projections without signal splitting can be generated invertedly. Furthermore, a penalized weighted least-square model with total variation(PWLSTV) is employed to denoise these projections, from which nPCT images with high accuracy are directly reconstructed.Numerical experiments demonstrate that this new algorithm is able to work with phase projections having any splitting distances. Moreover, results also reveal that nPCT images of higher signal-to-noise-ratio(SNR) could be reconstructed from projections having larger splitting distances. In summary, a novel model-driven nPCT image reconstruction algorithm with high accuracy and robustness is verified for the Lau interferometer-based hard x-ray nano-resolution phase contrast imaging.

Survey of methods and principles in three-dimensional reconstruction from two-dimensional medical images

Three-dimensional(3D)reconstruction of human organs has gained attention in recent years due to advances in the Internet and graphics processing units.In the coming years,most patient care will shift toward this new paradigm.However,development of fast and accurate 3D models from medical images or a set of medical scans remains a daunting task due to the number of pre-processing steps involved,most of which are dependent on human expertise.In this review,a survey of pre-processing steps was conducted,and reconstruction techniques for several organs in medical diagnosis were studied.Various methods and principles related to 3D reconstruction were highlighted.The usefulness of 3D reconstruction of organs in medical diagnosis was also highlighted.

Super-resolution reconstruction of synthetic-aperture radar image using adaptive-threshold singular value decomposition technique

A super-resolution reconstruction approach of （SVD） technique was presented, and its performance was radar image using an adaptive-threshold singular value decomposition analyzed, compared and assessed detailedly. First, radar imaging model and super-resolution reconstruction mechanism were outlined. Then, the adaptive-threshold SVD super-resolution algorithm, and its two key aspects, namely the determination method of point spread function （PSF） matrix T and the selection scheme of singular value threshold, were presented. Finally, the super-resolution algorithm was demonstrated successfully using the measured synthetic-aperture radar （SAR） images, and a Monte Carlo assessment was carried out to evaluate the performance of the algorithm by using the input/output signal-to-noise ratio （SNR）. Five versions of SVD algorithms, namely 1 ） using all singular values, 2） using the top 80% singular values, 3） using the top 50% singular values, 4） using the top 20% singular values and 5） using singular values s such that S2≥/max（s2）/rinsNR were tested. The experimental results indicate that when the singular value threshold is set as Smax/（rinSNR）1/2, the super-resolution algorithm provides a good compromise between too much noise and too much bias and has good reconstruction results.

Image super-resolution reconstruction based on sparse representation and residual compensation

A super-resolution reconstruction algorithm is proposed. The algorithm is based on the idea of the sparse representation of signals, by using the fact that the sparsest representation of a sig- nal is unique as the constraint of the patched-based reconstruction, and compensating residual errors of the reconstruction results both locally and globally to solve the distortion problem in patch-based reconstruction algorithms. Three reconstruction algorithms are compared. The results show that the images reconstructed with the new algorithm have the best quality.

Super-resolution image reconstruction based on three-step-training neural networks

A new method of super-resolution image reconstruction is proposed, which uses a three-step-training error backpropagation neural network （BPNN） to realize the super-resolution reconstruction （SRR） of satellite image. The method is based on BPNN. First, three groups learning samples with different resolutions are obtained according to image observation model, and then vector mappings are respectively used to those three group learning samples to speed up the convergence of BPNN, at last, three times consecutive training are carried on the BPNN. Training samples used in each step are of higher resolution than those used in the previous steps, so the increasing weights store a great amount of information for SRR, and network performance and generalization ability are improved greatly. Simulation and generalization tests are carried on the well-trained three-step-training NN respectively, and the reconstruction results with higher resolution images verify the effectiveness and validity of this method.

Multi-channel fast super-resolution image reconstruction based on matrix observation model

A multi-channel fast super-resolution image reconstruction algorithm based on matrix observation model is proposed in the paper,which consists of three steps to avoid the computational complexity: a single image SR reconstruction step,a registration step and a wavelet-based image fusion. This algorithm decomposes two large matrixes to the tensor product of two little matrixes and uses the natural isomorphism between matrix space and vector space to transform cost function based on matrix-vector products model to matrix form. Furthermore,we prove that the regularization part can be transformed to the matrix formed. The conjugate-gradient method is used to solve this new model. Finally,the wavelet fusion is used to integrate all the registered highresolution images obtained from the single image SR reconstruction step. The proposed algorithm reduces the storage requirement and the calculating complexity,and can be applied to large-dimension low-resolution images.

Transformer and GAN-Based Super-Resolution Reconstruction Network for Medical Images

Super-resolution reconstruction in medical imaging has become more demanding due to the necessity of obtaining high-quality images with minimal radiation dose,such as in low-field magnetic resonance imaging(MRI).However,image super-resolution reconstruction remains a difficult task because of the complexity and high textual requirements for diagnosis purpose.In this paper,we offer a deep learning based strategy for reconstructing medical images from low resolutions utilizing Transformer and generative adversarial networks(T-GANs).The integrated system can extract more precise texture information and focus more on important locations through global image matching after successfully inserting Transformer into the generative adversarial network for picture reconstruction.Furthermore,we weighted the combination of content loss,adversarial loss,and adversarial feature loss as the final multi-task loss function during the training of our proposed model T-GAN.In comparison to established measures like peak signal-to-noise ratio(PSNR)and structural similarity index measure(SSIM),our suggested T-GAN achieves optimal performance and recovers more texture features in super-resolution reconstruction of MRI scanned images of the knees and belly.

A generalized deep neural network approach for improving resolution of fluorescence microscopy images

Deep learning is capable of greatly promoting the progress of super-resolution imaging technology in terms of imaging and reconstruction speed,imaging resolution,and imagingflux.This paper proposes a deep neural network based on a generative adversarial network(GAN).The generator employs a U-Net-based network,which integrates Dense Net for the downsampling component.The proposed method has excellent properties,for example,the network model is trained with several different datasets of biological structures;the trained model can improve the imaging resolution of different microscopy imaging modalities such as confocal imaging and wide-field imaging;and the model demonstrates a generalized ability to improve the resolution of different biological structures even out of the datasets.In addition,experimental results showed that the method improved the resolution of caveolin-coated pits(CCPs)structures from 264 nm to 138 nm,a 1.91-fold increase,and nearly doubled the resolution of DNA molecules imaged while being transported through microfluidic channels.

Hyperspectral Image Super-Resolution Meets Deep Learning:A Survey and Perspective

Hyperspectral image super-resolution,which refers to reconstructing the high-resolution hyperspectral image from the input low-resolution observation,aims to improve the spatial resolution of the hyperspectral image,which is beneficial for subsequent applications.The development of deep learning has promoted significant progress in hyperspectral image super-resolution,and the powerful expression capabilities of deep neural networks make the predicted results more reliable.Recently,several latest deep learning technologies have made the hyperspectral image super-resolution method explode.However,a comprehensive review and analysis of the latest deep learning methods from the hyperspectral image super-resolution perspective is absent.To this end,in this survey,we first introduce the concept of hyperspectral image super-resolution and classify the methods from the perspectives with or without auxiliary information.Then,we review the learning-based methods in three categories,including single hyperspectral image super-resolution,panchromatic-based hyperspectral image super-resolution,and multispectral-based hyperspectral image super-resolution.Subsequently,we summarize the commonly used hyperspectral dataset,and the evaluations for some representative methods in three categories are performed qualitatively and quantitatively.Moreover,we briefly introduce several typical applications of hyperspectral image super-resolution,including ground object classification,urban change detection,and ecosystem monitoring.Finally,we provide the conclusion and challenges in existing learning-based methods,looking forward to potential future research directions.

Scene 3-D Reconstruction System in Scattering Medium

Research on neural radiance fields for novel view synthesis has experienced explosive growth with the development of new models and extensions.The NeRF(Neural Radiance Fields)algorithm,suitable for underwater scenes or scattering media,is also evolving.Existing underwater 3D reconstruction systems still face challenges such as long training times and low rendering efficiency.This paper proposes an improved underwater 3D reconstruction system to achieve rapid and high-quality 3D reconstruction.First,we enhance underwater videos captured by a monocular camera to correct the image quality degradation caused by the physical properties of the water medium and ensure consistency in enhancement across frames.Then,we perform keyframe selection to optimize resource usage and reduce the impact of dynamic objects on the reconstruction results.After pose estimation using COLMAP,the selected keyframes undergo 3D reconstruction using neural radiance fields(NeRF)based on multi-resolution hash encoding for model construction and rendering.In terms of image enhancement,our method has been optimized in certain scenarios,demonstrating effectiveness in image enhancement and better continuity between consecutive frames of the same data.In terms of 3D reconstruction,our method achieved a peak signal-to-noise ratio(PSNR)of 18.40 dB and a structural similarity(SSIM)of 0.6677,indicating a good balance between operational efficiency and reconstruction quality.

A rened analytical model for reconstruction problems in diuse reectance spectroscopy

A rened analytical model of spatially resolved diffuse reectance with small source-detector separations(SDSs)for the in vivo skin studies is proposed.Compared to the conventional model developed by Farrell et al.,it accounts for the limited acceptance angle of the detectorber.The rened model is validated in the wide range of optical parameters by Monte Carlo simulations of skin diffuse reectance at SDSs of units of mm.Cases of uniform dermis and two-layered epidermis-dermis structures are studied.Higher accuracy of the rened model compared to the conventional one is demonstrated in the separate,constraint-free reconstruction of absorption and reduced scattering spectra of uniform dermis from the Monte Carlo simulated data.In the case of epidermis-dermis geometry,the recovered values of reduced scattering in dermis are overestimated and the recovered values of absorption are underestimated for both analytical models.Presumably,in the presence of a thin mismatched topical layer,only the effective attenuation coe±cient of the bottom layer can be accurately recovered using a diffusion theorybased analytical model while separate reconstruction of absorption and reduced scattering fails due to the inapplicability of the method of images.These-ndings require implementation of more sophisticated models of light transfer in inhomogeneous media in the recovery algorithms.

Evaluating the use of three-dimensional reconstruction visualization technology for precise laparoscopic resection in gastroesophageal junction cancer

BACKGROUND Laparoscopic gastrectomy for esophagogastric junction(EGJ)carcinoma enables the removal of the carcinoma at the junction between the stomach and esophagus while preserving the gastric function,thereby providing patients with better treatment outcomes and quality of life.Nonetheless,this surgical technique also presents some challenges and limitations.Therefore,three-dimensional reconstruction visualization technology(3D RVT)has been introduced into the procedure,providing doctors with more comprehensive and intuitive anatomical information that helps with surgical planning,navigation,and outcome evaluation.AIM To discuss the application and advantages of 3D RVT in precise laparoscopic resection of EGJ carcinomas.METHODS Data were obtained from the electronic or paper-based medical records at The First Affiliated Hospital of Hebei North University from January 2020 to June 2022.A total of 120 patients diagnosed with EGJ carcinoma were included in the study.Of these,68 underwent laparoscopic resection after computed tomography(CT)-enhanced scanning and were categorized into the 2D group,whereas 52 underwent laparoscopic resection after CT-enhanced scanning and 3D RVT and were categorized into the 3D group.This study had two outcome measures:the deviation between tumor-related factors(such as maximum tumor diameter and infiltration length)in 3D RVT and clinical reality,and surgical outcome indicators(such as operative time,intraoperative blood loss,number of lymph node dissections,R0 resection rate,postoperative hospital stay,postoperative gas discharge time,drainage tube removal time,and related complications)between the 2D and 3D groups.RESULTS Among patients included in the 3D group,27 had a maximum tumor diameter of less than 3 cm,whereas 25 had a diameter of 3 cm or more.In actual surgical observations,24 had a diameter of less than 3 cm,whereas 28 had a diameter of 3 cm or more.The findings were consistent between the two methods(χ^(2)=0.346,P=0.556),with a kappa consistency coefficient of 0.808.With respect to infiltration length,in the 3D group,23 patients had a length of less than 5 cm,whereas 29 had a length of 5 cm or more.In actual surgical observations,20 cases had a length of less than 5 cm,whereas 32 had a length of 5 cm or more.The findings were consistent between the two methods(χ^(2)=0.357,P=0.550),with a kappa consistency coefficient of 0.486.Pearson correlation analysis showed that the maximum tumor diameter and infiltration length measured using 3D RVT were positively correlated with clinical observations during surgery(r=0.814 and 0.490,both P<0.05).The 3D group had a shorter operative time(157.02±8.38 vs 183.16±23.87),less intraoperative blood loss(83.65±14.22 vs 110.94±22.05),and higher number of lymph node dissections(28.98±2.82 vs 23.56±2.77)and R0 resection rate(80.77%vs 61.64%)than the 2D group.Furthermore,the 3D group had shorter hospital stay[8(8,9)vs 13(14,16)],time to gas passage[3(3,4)vs 4(5,5)],and drainage tube removal time[4(4,5)vs 6(6,7)]than the 2D group.The complication rate was lower in the 3D group(11.54%)than in the 2D group(26.47%)(χ^(2)=4.106,P<0.05).CONCLUSION Using 3D RVT,doctors can gain a more comprehensive and intuitive understanding of the anatomy and related lesions of EGJ carcinomas,thus enabling more accurate surgical planning.

GPU-accelerated three-dimensional reconstruction method of the Compton camera and its application in radionuclide imaging

A novel and fast three-dimensional reconstruction method for a Compton camera and its performance in radionuclide imaging is proposed and analyzed in this study. The conical surface sampling back-projection method with scattering angle correction(CSS-BP-SC) can quickly perform the back-projection process of the Compton cone and can be used to precompute the list-mode maximum likelihood expectation maximization(LM-MLEM). A dedicated parallel architecture was designed for the graphics processing unit acceleration of the back-projection and iteration stage of the CSS-BP-SC-based LM-MLEM. The imaging results of the two-point source Monte Carlo(MC) simulation demonstrate that by analyzing the full width at half maximum along the three coordinate axes, the CSS-BP-SC-based LM-MLEM can obtain imaging results comparable to those of the traditional reconstruction algorithm, that is, the simple back-projection-based LM-MLEM. The imaging results of the mouse phantom MC simulation and experiment demonstrate that the reconstruction results obtained by the proposed method sufficiently coincide with the set radioactivity distribution, and the speed increased by more than 664 times compared to the traditional reconstruction algorithm in the mouse phantom experiment. The proposed method will further advance the imaging applications of Compton cameras.

Deep learning-based magnetic resonance imaging reconstruction for improving the image quality of reduced-field-of-view diffusionweighted imaging of the pancreas

BACKGROUND It has been reported that deep learning-based reconstruction(DLR)can reduce image noise and artifacts,thereby improving the signal-to-noise ratio and image sharpness.However,no previous studies have evaluated the efficacy of DLR in improving image quality in reduced-field-of-view(reduced-FOV)diffusionweighted imaging(DWI)[field-of-view optimized and constrained undistorted single-shot(FOCUS)]of the pancreas.We hypothesized that a combination of these techniques would improve DWI image quality without prolonging the scan time but would influence the apparent diffusion coefficient calculation.AIM To evaluate the efficacy of DLR for image quality improvement of FOCUS of the pancreas.METHODS This was a retrospective study evaluated 37 patients with pancreatic cystic lesions who underwent magnetic resonance imaging between August 2021 and October 2021.We evaluated three types of FOCUS examinations:FOCUS with DLR(FOCUS-DLR+),FOCUS without DLR(FOCUS-DLR−),and conventional FOCUS(FOCUS-conv).The three types of FOCUS and their apparent diffusion coefficient(ADC)maps were compared qualitatively and quantitatively.RESULTS FOCUS-DLR+(3.62,average score of two radiologists)showed significantly better qualitative scores for image noise than FOCUS-DLR−(2.62)and FOCUS-conv(2.88)(P<0.05).Furthermore,FOCUS-DLR+showed the highest contrast ratio and 600 s/mm^(2)(0.72±0.08 and 0.68±0.08)and FOCUS-DLR−showed the highest CR between cystic lesions and the pancreatic parenchyma for the b-values of 0 and 600 s/mm2(0.62±0.21 and 0.62±0.21)(P<0.05),respectively.FOCUS-DLR+provided significantly higher ADCs of the pancreas and lesion(1.44±0.24 and 3.00±0.66)compared to FOCUS-DLR−(1.39±0.22 and 2.86±0.61)and significantly lower ADCs compared to FOCUS-conv(1.84±0.45 and 3.32±0.70)(P<0.05),respectively.CONCLUSION This study evaluated the efficacy of DLR for image quality improvement in reduced-FOV DWI of the pancreas.DLR can significantly denoise images without prolonging the scan time or decreasing the spatial resolution.The denoising level of DWI can be controlled to make the images appear more natural to the human eye.However,this study revealed that DLR did not ameliorate pancreatic distortion.Additionally,physicians should pay attention to the interpretation of ADCs after DLR application because ADCs are significantly changed by DLR.

IRMIRS:Inception-ResNet-Based Network for MRI Image Super-Resolution

Medical image super-resolution is a fundamental challenge due to absorption and scattering in tissues.These challenges are increasing the interest in the quality of medical images.Recent research has proven that the rapid progress in convolutional neural networks(CNNs)has achieved superior performance in the area of medical image super-resolution.However,the traditional CNN approaches use interpolation techniques as a preprocessing stage to enlarge low-resolution magnetic resonance(MR)images,adding extra noise in the models and more memory consumption.Furthermore,conventional deep CNN approaches used layers in series-wise connection to create the deeper mode,because this later end layer cannot receive complete information and work as a dead layer.In this paper,we propose Inception-ResNet-based Network for MRI Image Super-Resolution known as IRMRIS.In our proposed approach,a bicubic interpolation is replaced with a deconvolution layer to learn the upsampling filters.Furthermore,a residual skip connection with the Inception block is used to reconstruct a high-resolution output image from a low-quality input image.Quantitative and qualitative evaluations of the proposed method are supported through extensive experiments in reconstructing sharper and clean texture details as compared to the state-of-the-art methods.

Digital holographic imaging via direct quantum wavefunction reconstruction

Wavefunction is a fundamental concept of quantum theory.Recent studies have shown surprisingly that wavefunction can be directly reconstructed via the measurement of weak value.The weak value based direct wavefunction reconstruction not only gives the operational meaning of wavefunction,but also provides the possibility of realizing holographic imaging with a totally new quantum approach.Here,we review the basic background knowledge of weak value based direct wavefunction reconstruction combined with recent experimental demonstrations.The main purpose of this work focuses on the idea of holographic imaging via direct wavefunction reconstruction.Since research on this topic is still in its early stage,we hope that this work can attract interest in the field of traditional holographic imaging.In addition,the wavefunction holographic imaging may find important applications in quantum information science.