Estimation-free spatial-domain image reconstruction of structured illumination microscopy

Structured illumination microscopy(SIM)achieves super-resolution(SR)by modulating the high-frequency information of the sample into the passband of the optical system and subsequent image reconstruction.The traditional Wiener-filtering-based reconstruction algorithm operates in the Fourier domain,it requires prior knowledge of the sinusoidal illumination patterns which makes the time-consuming procedure of parameter estimation to raw datasets necessary,besides,the parameter estimation is sensitive to noise or aberration-induced pattern distortion which leads to reconstruction artifacts.Here,we propose a spatial-domain image reconstruction method that does not require parameter estimation but calculates patterns from raw datasets,and a reconstructed image can be obtained just by calculating the spatial covariance of differential calculated patterns and differential filtered datasets(the notch filtering operation is performed to the raw datasets for attenuating and compensating the optical transfer function(OTF)).Experiments on reconstructing raw datasets including nonbiological,biological,and simulated samples demonstrate that our method has SR capability,high reconstruction speed,and high robustness to aberration and noise.

Triple-path feature transform network for ring-array photoacoustic tomography image reconstruction

Photoacoustic imaging(PAI)is a noninvasive emerging imaging method based on the photoacoustic effect,which provides necessary assistance for medical diagnosis.It has the characteristics of large imaging depth and high contrast.However,limited by the equipment cost and reconstruction time requirements,the existing PAI systems distributed with annular array transducers are difficult to take into account both the image quality and the imaging speed.In this paper,a triple-path feature transform network(TFT-Net)for ring-array photoacoustic tomography is proposed to enhance the imaging quality from limited-view and sparse measurement data.Specifically,the network combines the raw photoacoustic pressure signals and conventional linear reconstruction images as input data,and takes the photoacoustic physical model as a prior information to guide the reconstruction process.In addition,to enhance the ability of extracting signal features,the residual block and squeeze and excitation block are introduced into the TFT-Net.For further efficient reconstruction,the final output of photoacoustic signals uses‘filter-then-upsample’operation with a pixel-shuffle multiplexer and a max out module.Experiment results on simulated and in-vivo data demonstrate that the constructed TFT-Net can restore the target boundary clearly,reduce background noise,and realize fast and high-quality photoacoustic image reconstruction of limited view with sparse sampling.

Multi-scale cross-domain alignment for person image generation

Person image generation aims to generate images that maintain the original human appearance in different target poses.Recent works have revealed that the critical element in achieving this task is the alignment of appearance domain and pose domain.Previous alignment methods,such as appearance flow warping,correspondence learning and cross attention,often encounter challenges when it comes to producing fine texture details.These approaches suffer from limitations in accurately estimating appearance flows due to the lack of global receptive field.Alternatively,they can only perform cross-domain alignment on high-level feature maps with small spatial dimensions since the computational complexity increases quadratically with larger feature sizes.In this article,the significance of multi-scale alignment,in both low-level and high-level domains,for ensuring reliable cross-domain alignment of appearance and pose is demonstrated.To this end,a novel and effective method,named Multi-scale Crossdomain Alignment(MCA)is proposed.Firstly,MCA adopts global context aggregation transformer to model multi-scale interaction between pose and appearance inputs,which employs pair-wise window-based cross attention.Furthermore,leveraging the integrated global source information for each target position,MCA applies flexible flow prediction head and point correlation to effectively conduct warping and fusing for final transformed person image generation.Our proposed MCA achieves superior performance on two popular datasets than other methods,which verifies the effectiveness of our approach.

A new imaging mode based on X-ray CT as prior image and sparsely sampled projections for rapid clinical proton CT

Proton computed tomography(CT)has a distinct practical significance in clinical applications.It eliminates 3–5%errors caused by the transformation of Hounsfield unit(HU)to relative stopping power(RSP)values when using X-ray CT for positioning and treatment planning systems(TPSs).Following the development of FLASH proton therapy,there are increased requirements for accurate and rapid positioning in TPSs.Thus,a new rapid proton CT imaging mode is proposed based on sparsely sampled projections.The proton beam was boosted to 350 MeV by a compact proton linear accelerator(LINAC).In this study,the comparisons of the proton scattering with the energy of 350 MeV and 230 MeV are conducted based on GEANT4 simulations.As the sparsely sampled information associated with beam acquisitions at 12 angles is not enough for reconstruction,X-ray CT is used as a prior image.The RSP map generated by converting the X-ray CT was constructed based on Monte Carlo simulations.Considering the estimation of the most likely path(MLP),the prior image-constrained compressed sensing(PICCS)algorithm is used to reconstruct images from two different phantoms using sparse proton projections of 350 MeV parallel proton beam.The results show that it is feasible to realize the proton image reconstruction with the rapid proton CT imaging proposed in this paper.It can produce RSP maps with much higher accuracy for TPSs and fast positioning to achieve ultra-fast imaging for real-time image-guided radiotherapy(IGRT)in clinical proton therapy applications.

Artificial Intelligence-Based Image Reconstruction for Computed Tomography: A Survey

Computed tomography has made significant advances since its intro-duction in the early 1970s,where researchers have mainly focused on the quality of image reconstruction in the early stage.However,radiation exposure poses a health risk,prompting the demand of the lowest possible dose when carrying out CT examinations.To acquire high-quality reconstruction images with low dose radiation,CT reconstruction techniques have evolved from conventional reconstruction such as analytical and iterative reconstruction,to reconstruction methods based on artificial intelligence(AI).All these efforts are devoted to con-structing high-quality images using only low doses with fast reconstruction speed.In particular,conventional reconstruction methods usually optimize one aspect,while AI-based reconstruction has finally managed to attain all goals in one shot.However,there are limitations such as the requirements on large datasets,unstable performance,and weak generalizability in AI-based reconstruction methods.This work presents the review and discussion on the classification,the commercial use,the advantages,and the limitations of AI-based image reconstruction methods in CT.

Image reconstruction based on total-variation minimization and alternating direction method in linear scan computed tomography

Linear scan computed tomography （LCT） is of great benefit to online industrial scanning and security inspection due to its characteristics of straight-line source trajectory and high scanning speed. However, in practical applications of LCT, there are challenges to image reconstruction due to limited-angle and insufficient data. In this paper, a new reconstruction algorithm based on total-variation （TV） minimization is developed to reconstruct images from limited-angle and insufficient data in LCT. The main idea of our approach is to reformulate a TV problem as a linear equality constrained problem where the objective function is separable, and then minimize its augmented Lagrangian function by using alternating direction method （ADM） to solve subproblems. The proposed method is robust and efficient in the task of reconstruction by showing the convergence of ADM. The numerical simulations and real data reconstructions show that the proposed reconstruction method brings reasonable performance and outperforms some previous ones when applied to an LCT imaging problem.

An algorithm for computed tomography image reconstruction from limited-view projections

With the development of the compressive sensing theory, the image reconstruction from the projections viewed in limited angles is one of the hot problems in the research of computed tomography technology. This paper develops an iterative algorithm for image reconstruction, which can fit the most cases. This method gives an image reconstruction flow with the difference image vector, which is based on the concept that the difference image vector between the reconstructed and the reference image is sparse enough. Then the l1-norm minimization method is used to reconstruct the difference vector to recover the image for flat subjects in limited angles. The algorithm has been tested with a thin planar phantom and a real object in limited-view projection data. Moreover, all the studies showed the satisfactory results in accuracy at a rather high reconstruction speed.

Reconstruction of electrical capacitance tomography images based on fast linearized alternating direction method of multipliers for two-phase flow system

Electrical capacitance tomography(ECT)has been applied to two-phase flow measurement in recent years.Image reconstruction algorithms play an important role in the successful applications of ECT.To solve the ill-posed and nonlinear inverse problem of ECT image reconstruction,a new ECT image reconstruction method based on fast linearized alternating direction method of multipliers(FLADMM)is proposed in this paper.On the basis of theoretical analysis of compressed sensing(CS),the data acquisition of ECT is regarded as a linear measurement process of permittivity distribution signal of pipe section.A new measurement matrix is designed and L1 regularization method is used to convert ECT inverse problem to a convex relaxation problem which contains prior knowledge.A new fast alternating direction method of multipliers which contained linearized idea is employed to minimize the objective function.Simulation data and experimental results indicate that compared with other methods,the quality and speed of reconstructed images are markedly improved.Also,the dynamic experimental results indicate that the proposed algorithm can ful fill the real-time requirement of ECT systems in the application.

A new inversion method for reconstruction of plasmaspheric He^(+)density from EUV images

The Computer Tomography(CT)method is used for remote sensing the Earth’s plasmasphere.One challenge for image reconstruction is insufficient projection data,mainly caused by limited projection angles.In this study,we apply the Algebraic Reconstruction Technique(ART)and the minimization of the image Total Variation(TV)method,with a combination of priori knowledge of north–south symmetry,to reconstruct plasmaspheric He+density from simulated EUV images.The results demonstrate that incorporating priori assumption can be particularly useful when the projection data is insufficient.This method has good performance even with a projection angle of less than 150 degrees.The method of our study is expected to have applications in the Soft X-ray Imager(SXI)reconstruction for the Solar wind–Magnetosphere–Ionosphere Link Explorer(SMILE)mission.

Hybrid model for muon tomography and quantitative analysis of image quality

Muon tomography is a novel method for the non-destructive imaging of materials based on muon rays,which are highly penetrating in natural background radiation.Currently,the most commonly used imaging methods include muon radiography and muon tomography.A previously studied method known as coinciding muon trajectory density tomography,which utilizes muonic secondary particles,is proposed to image low and medium atomic number(Z)materials.However,scattering tomography is mostly used to image high-Z materials,and coinciding muon trajectory density tomography exhibits a hollow phenomenon in the imaging results owing to the self-absorption effect.To address the shortcomings of the individual imaging methods,hybrid model tomography combining scattering tomography and coinciding muon trajectory density tomography is proposed and verified.In addition,the peak signal-to-noise ratio was introduced to quantitatively analyze the image quality.Different imaging models were simulated using the Geant4 toolkit to confirm the advantages of this innovative method.The simulation results showed that hybrid model tomography can image centimeter-scale materials with low,medium,and high Z simultaneously.For high-Z materials with similar atomic numbers,this method can clearly distinguish those with apparent differences in density.According to the peak signal-to-noise ratio of the analysis,the reconstructed image quality of the new method was significantly higher than that of the individual imaging methods.This study provides a reliable approach to the compatibility of scattering tomography and coinciding muon trajectory density tomography.

Image reconstruction for cone-beam computed tomography using total p-variation plus Kullback-Leibler data divergence

Accurate reconstruction from a reduced data set is highly essential for computed tomography in fast and/or low dose imaging applications. Conventional total variation（TV）-based algorithms apply the L1 norm-based penalties, which are not as efficient as Lp（0〈p〈1） quasi-norm-based penalties. TV with a p-th power-based norm can serve as a feasible alternative of the conventional TV, which is referred to as total p-variation（TpV）. This paper proposes a TpV-based reconstruction model and develops an efficient algorithm. The total p-variation and Kullback-Leibler（KL） data divergence, which has better noise suppression capability compared with the often-used quadratic term, are combined to build the reconstruction model. The proposed algorithm is derived by the alternating direction method（ADM） which offers a stable, efficient, and easily coded implementation. We apply the proposed method in the reconstructions from very few views of projections（7 views evenly acquired within 180°）. The images reconstructed by the new method show clearer edges and higher numerical accuracy than the conventional TV method. Both the simulations and real CT data experiments indicate that the proposed method may be promising for practical applications.

Evolutionary Computation Based Optimization of Image Zernike Moments Shape Feature Vector

The image shape feature can be described by the image Zernike moments. In this paper, we points out the problem that the high dimension image Zernike moments shape feature vector can describe more detail of the original image but has too many elements making trouble for the next image analysis phases. Then the low dimension image Zernike moments shape feature vector should be improved and optimized to describe more detail of the original image. So the optimization algorithm based on evolutionary computation is designed and implemented in this paper to solve this problem. The experimental results demonstrate the feasibility of the optimization algorithm.

Deep learning approach to detect seizure using reconstructed phase space images

Epilepsy is a chronic neurological disorder that affects the function of the brain in people of all ages.It manifests in the electroencephalogram(EEG) signal which records the electrical activity of the brain.Various image processing,signal processing,and machine-learning based techniques are employed to analyze epilepsy,using spatial and temporal features.The nervous system that generates the EEG signal is considered nonlinear and the EEG signals exhibit chaotic behavior.In order to capture these nonlinear dynamics,we use reconstructed phase space(RPS) representation of the signal.Earlier studies have primarily addressed seizure detection as a binary classification(normal vs.ictal) problem and rarely as a ternary class(normal vs.interictal vs.ictal)problem.We employ transfer learning on a pre-trained deep neural network model and retrain it using RPS images of the EEG signal.The classification accuracy of the model for the binary classes is(98.5±1.5)% and(95±2)% for the ternary classes.The performance of the convolution neural network(CNN) model is better than the other existing statistical approach for all performance indicators such as accuracy,sensitivity,and specificity.The result of the proposed approach shows the prospect of employing RPS images with CNN for predicting epileptic seizures.

Optimization-based image reconstruction in x-ray computed tomography by sparsity exploitation of local continuity and nonlocal spatial self-similarity

The additional sparse prior of images has been the subject of much research in problems of sparse-view computed tomography（CT） reconstruction. A method employing the image gradient sparsity is often used to reduce the sampling rate and is shown to remove the unwanted artifacts while preserve sharp edges, but may cause blocky or patchy artifacts.To eliminate this drawback, we propose a novel sparsity exploitation-based model for CT image reconstruction. In the presented model, the sparse representation and sparsity exploitation of both gradient and nonlocal gradient are investigated.The new model is shown to offer the potential for better results by introducing a similarity prior information of the image structure. Then, an effective alternating direction minimization algorithm is developed to optimize the objective function with a robust convergence result. Qualitative and quantitative evaluations have been carried out both on the simulation and real data in terms of accuracy and resolution properties. The results indicate that the proposed method can be applied for achieving better image-quality potential with the theoretically expected detailed feature preservation.

Image Reconstruction of Ghost Imaging Based on Improved Generative Adversarial Networks

In this paper, we improve traditional generative adversarial networks (GAN) with reference to residual networks and convolutional neural networks to facilitate the reconstruction of complex objects that cannot be reconstructed by traditional associative imaging methods. Unlike traditional ghost imaging to reconstruct objects from bucket signals, our proposed method can use simple objects (such as EMNIST) as a training set for GAN, and then recognize objects (such as faces) of completely different complexity than the training set. We use traditional ghost imaging and neural network to reconstruct target objects respectively. According to the research results in this paper, the method based on neural network can reconstruct complex objects very well, but the method based on traditional ghost imaging cannot reconstruct complex objects. The research scheme in this paper is of great significance for the reconstruction of complex object-related imaging under low sampling conditions.

A Compton scattering image reconstruction algorithm based on total variation minimization

Compton scattering imaging is a novel radiation imaging method using scattered photons.Its main characteristics are detectors that do not have to be on the opposite side of the source,so avoiding the rotation process.The reconstruction problem of Compton scattering imaging is the inverse problem to solve electron densities from nonlinear equations,which is ill-posed.This means the solution exhibits instability and sensitivity to noise or erroneous measurements.Using the theory for reconstruction of sparse images,a reconstruction algorithm based on total variation minimization is proposed.The reconstruction problem is described as an optimization problem with nonlinear data-consistency constraint.The simulated results show that the proposed algorithm could reduce reconstruction error and improve image quality,especially when there are not enough measurements.

Wavelet-Based Mixed-Resolution Coding Approach Incorporating with SPT for the Stereo Image

With the advances of display technology, three-dimensional(3-D) imaging systems are becoming increasingly popular. One way of stimulating 3-D perception is to use stereo pairs, a pair of images of the same scene acquired from different perspectives. Since there is an inherent redundancy between the images of a stereo pairs, data compression algorithms should be employed to represent stereo pairs efficiently. The proposed techniques generally use block-based disparity compensation. In order to get the higher compression ratio, this paper employs the wavelet-based mixed-resolution coding technique to incorporate with SPT-based disparity-compensation to compress the stereo image data. The mixed-resolution coding is a perceptually justified technique that is achieved by presenting one eye with a low-resolution image and the other with a high-resolution image. Psychophysical experiments show that the stereo image pairs with one high-resolution image and one low-resolution image provide almost the same stereo depth to that of a stereo image with two high-resolution images. By combining the mixed-resolution coding and SPT-based disparity-compensation techniques, one reference (left) high-resolution image can be compressed by a hierarchical wavelet transform followed by vector quantization and Huffman encoder. After two level wavelet decompositions, for the low-resolution right image and low-resolution left image, subspace projection technique using the fixed block size disparity compensation estimation is used. At the decoder, the low-resolution right subimage is estimated using the disparity from the low-resolution left subimage. A full-size reconstruction is obtained by upsampling a factor of 4 and reconstructing with the synthesis low pass filter. Finally, experimental results are presented, which show that our scheme achieves a PSNR gain (about 0.92dB) as compared to the current block-based disparity compensation coding techniques.

Super-resolution image reconstruction based on three-step-training neural networks

A new method of super-resolution image reconstruction is proposed, which uses a three-step-training error backpropagation neural network （BPNN） to realize the super-resolution reconstruction （SRR） of satellite image. The method is based on BPNN. First, three groups learning samples with different resolutions are obtained according to image observation model, and then vector mappings are respectively used to those three group learning samples to speed up the convergence of BPNN, at last, three times consecutive training are carried on the BPNN. Training samples used in each step are of higher resolution than those used in the previous steps, so the increasing weights store a great amount of information for SRR, and network performance and generalization ability are improved greatly. Simulation and generalization tests are carried on the well-trained three-step-training NN respectively, and the reconstruction results with higher resolution images verify the effectiveness and validity of this method.

Method of lateral image reconstruction in structured illumination microscopy with super resolution

The image reconstruction process in super-resolution structured illumination microscopy(SIM)is investigated.The structured pattern is generated by the interference of two Gaussian beams to encode undetectable spectra into detectable region of microscope.After parameters estimation of the structured pattern,the encoded spectra are computationally decoded and recombined in Fourier domain to equivalently increase the cut-off frequency of microscope,resulting in the extension of detectable spectra and a reconstructed image with about two-fold enhanced resolution.Three di®erent methods to estimate the initial phase of structured pattern are compared,verifying the auto-correlation algorithm a®ords the fast,most precise and robust measurement.The artifacts sources and detailed reconstruction°owchart for both linear and nonlinear SIM are also presented.

Frequency domain based super-resolution method for mixed-resolution multi-view images

Super-resolution (SR) techniques, which are based on single or multi-frame low-resolution (LR) images, have been extensively investigated in the last two decades. Mixed-resolution multiview video format plays an important role in three-dimensional television (3DTV) coding scheme. Previous work considers multiview or multi-camera images and videos at the same resolution, which performs well under the planar model without or with little projection error among the videos captured by different cameras. In recent years, several researchers have discussed the SR problem in mixed-resolution multi-view video format, where the superresolved image is created using the up-sampled version of the LR image and the high frequency components extracted from the warped image in the adjacent high-resolution (HR) views. Unfortunately, the output HR images suffer from artifacts caused by depth error. To obtain the detailed texture and edge information from the HR image as much as possible, while preserving the structure of the LR image, a novel SR reconstruction algorithm is proposed. The algorithm is composed of three components: the structure term, the detail information term, and the regularization term. The first term preserves the structure similarity of the LR image; the second term extracts detailed information from the adjacent HR image; and the last term ensures the uniqueness of the solution. Experimental results show the effectiveness and robustness of the proposed algorithm, which achieves high performance both subjectively and objectively. © 2016 Beijing Institute of Aerospace Information.