GAN-DIRNet:A Novel Deformable Image Registration Approach for Multimodal Histological Images

Multi-modal histological image registration tasks pose significant challenges due to tissue staining operations causing partial loss and folding of tissue.Convolutional neural network(CNN)and generative adversarial network(GAN)are pivotal inmedical image registration.However,existing methods often struggle with severe interference and deformation,as seen in histological images of conditions like Cushing’s disease.We argue that the failure of current approaches lies in underutilizing the feature extraction capability of the discriminator inGAN.In this study,we propose a novel multi-modal registration approach GAN-DIRNet based on GAN for deformable histological image registration.To begin with,the discriminators of two GANs are embedded as a new dual parallel feature extraction module into the unsupervised registration networks,characterized by implicitly extracting feature descriptors of specific modalities.Additionally,modal feature description layers and registration layers collaborate in unsupervised optimization,facilitating faster convergence and more precise results.Lastly,experiments and evaluations were conducted on the registration of the Mixed National Institute of Standards and Technology database(MNIST),eight publicly available datasets of histological sections and the Clustering-Registration-Classification-Segmentation(CRCS)dataset on the Cushing’s disease.Experimental results demonstrate that our proposed GAN-DIRNet method surpasses existing approaches like DIRNet in terms of both registration accuracy and time efficiency,while also exhibiting robustness across different image types.

Nonlinear Registration of Brain Magnetic Resonance Images with Cross Constraints of Intensity and Structure

Many deep learning-based registration methods rely on a single-stream encoder-decoder network for computing deformation fields between 3D volumes.However,these methods often lack constraint information and overlook semantic consistency,limiting their performance.To address these issues,we present a novel approach for medical image registration called theDual-VoxelMorph,featuring a dual-channel cross-constraint network.This innovative network utilizes both intensity and segmentation images,which share identical semantic information and feature representations.Two encoder-decoder structures calculate deformation fields for intensity and segmentation images,as generated by the dual-channel cross-constraint network.This design facilitates bidirectional communication between grayscale and segmentation information,enabling the model to better learn the corresponding grayscale and segmentation details of the same anatomical structures.To ensure semantic and directional consistency,we introduce constraints and apply the cosine similarity function to enhance semantic consistency.Evaluation on four public datasets demonstrates superior performance compared to the baselinemethod,achieving Dice scores of 79.9%,64.5%,69.9%,and 63.5%for OASIS-1,OASIS-3,LPBA40,and ADNI,respectively.

Automated Registration for Infrared Image Based on Wavelet Analysis

To develop a quick, accurate and antinoise automated image registration technique for infrared images, the wavelet analysis technique was used to extract the feature points in two images followed by the compensation for input image with angle difference between them. A hi erarchical feature matching algorithm was adopted to get the final transform parameters between the two images. The simulation results for two infrared images show that the method can effectively, quickly and accurately register images and be antinoise to some extent.

Projection registration of X-ray image and CT image

A methodology for alignment of an X-ray image and a CT image, based on the Chamfer 3-4 distance transform and simulated annealing optimization algorithm is presented. Firstly, an initial transformation matrix is constructed. For the convenience of computing, geometric models of the X-ray device to reconstruct the calibration matrix are used. Then, by defining the distance between the 3-D protective and the 2-D object image, we optimize this distance matching problem, using the simulated annealing algorithm. This method is also integrated into medical intra-operation, dealing with the data set acquired from 3-D image workstation and active navigation.

IMAGE PANORAMIC MOS AICING WITH GLOBAL AND LOCAL REGISTRATION

Technique s for constructing full view panoramic mosaics from sequences of images are pres ented. The goal of this work is to remove too many limitations for pure panning motion. The best reference block is important for the block-matching method for improving the robustness and performance. It is automatically selected in the h igh-frequency image, which always contains the plenty visible features. In orde r to reduce accumulated registration errors, the global registration using the p hase-correlation matching method with rotation adjustment is applied to the who le sequence of images, which results in an optimal image mosaic with resolving t ranslational or rotational motion. The local registration using the Levenberg-M arquardt iterative non-linear minimization algorithm is applied to compensating for small amounts of motion parallax introduced by translations of the camera a nd other unmodeled distortions, then minimizing the discrepancy after applying t he global registration. The accumulated misregistration errors may cause a visib le gap between the two images. A smoothing filter is introduced for removing the visible artifact.

Novel registration algorithm for 3-D images captured from multiple views of object surface

A novel algorithm of 3-D surface image registration is proposed. It makes use of the array information of 3-D points and takes vector/vertex-like features as the basis of the matching. That array information of 3-D points can be easily obtained when capturing original 3-D images. The iterative least-mean-squared （LMS） algorithm is applied to optimizing adaptively the transformation matrix parameters. These can effectively improve the registration performance and hurry up the matching process. Experimental results show that it can reach a good subjective impression on aligned 3-D images. Although the algorithm focuses primarily on the human head model, it can also be used for other objects with small modifications.

A method based on mutual information and gradient information for medical image registration

Mutual information is widely used in medical image registration, because it does not require preprocessing the image. However, the local maximum problem in the registration is insurmountable. We combine mutual information and gradient information to solve this problem and apply it to the non-rigid deformation image registration. To improve the accuracy, we provide some implemental issues, for example, the Powell searching algorithm, gray interpolation and consideration of outlier points. The experimental results show the accuracy of the method and the feasibility in non-rigid medical image registration.

A Review of Point Feature Based Medical Image Registration

Point features, as the basis of lines, surfaces, and bodies, are commonly used in medical image registration. To obtain an elegant spatial transformation of extracted feature points, many point set matching algorithms(PMs) have been developed to match two point sets by optimizing multifarious distance functions. There are ample reviews related to medical image registration and PMs which summarize their basic principles and main algorithms separately. However,to data, detailed summary of PMs used in medical image registration in different clinical environments has not been published. In this paper, we provide a comprehensive review of the existing key techniques of the PMs applied to medical image registration according to the basic principles and clinical applications. As the core technique of the PMs, geometric transformation models are elaborated in this paper, demonstrating the mechanism of point set registration. We also focus on the clinical applications of the PMs and propose a practical classification method according to their applications in different clinical surgeries. The aim of this paper is to provide a summary of pointfeaturebased methods used in medical image registration and to guide doctors or researchers interested in this field to choose appropriate techniques in their research.

SuperFusion: A Versatile Image Registration and Fusion Network with Semantic Awareness

Image fusion aims to integrate complementary information in source images to synthesize a fused image comprehensively characterizing the imaging scene. However, existing image fusion algorithms are only applicable to strictly aligned source images and cause severe artifacts in the fusion results when input images have slight shifts or deformations. In addition,the fusion results typically only have good visual effect, but neglect the semantic requirements of high-level vision tasks.This study incorporates image registration, image fusion, and semantic requirements of high-level vision tasks into a single framework and proposes a novel image registration and fusion method, named Super Fusion. Specifically, we design a registration network to estimate bidirectional deformation fields to rectify geometric distortions of input images under the supervision of both photometric and end-point constraints. The registration and fusion are combined in a symmetric scheme, in which while mutual promotion can be achieved by optimizing the naive fusion loss, it is further enhanced by the mono-modal consistent constraint on symmetric fusion outputs. In addition, the image fusion network is equipped with the global spatial attention mechanism to achieve adaptive feature integration. Moreover, the semantic constraint based on the pre-trained segmentation model and Lovasz-Softmax loss is deployed to guide the fusion network to focus more on the semantic requirements of high-level vision tasks. Extensive experiments on image registration, image fusion,and semantic segmentation tasks demonstrate the superiority of our Super Fusion compared to the state-of-the-art alternatives.The source code and pre-trained model are publicly available at https://github.com/Linfeng-Tang/Super Fusion.

Using shape contexts method for registration of contra lateral breasts in thermal images

AIM: To achieve symmetric boundaries for left and right breasts boundaries in thermal images by registration. METHODS: The proposed method for registration consists of two steps. In the first step, shape context, an approach as presented by Belongie and Malik was applied for registration of two breast boundaries. The shape context is an approach to measure shape similarity. Two sets of finite sample points from shape contours of two breasts are then presented. Consequently, the correspondences between the two shapes are found. By finding correspondences, the sample point which has the most similar shape context is obtained. RESULTS: In this study, a line up transformation which maps one shape onto the other has been estimated in order to complete shape. The used of a thin plate spline permitted good estimation of a plane transformation which has capability to map unselective points from one shape onto the other. The obtained aligningtransformation of boundaries points has been applied successfully to map the two breasts interior points. Some of advantages for using shape context method in this work are as follows:(1) no special land marks or key points are needed;(2) it is tolerant to all common shape deformation; and(3) although it is uncomplicated and straightforward to use, it gives remarkably powerful descriptor for point sets significantly upgrading point set registration. Results are very promising. The proposed algorithm was implemented for 32 cases. Boundary registration is done perfectly for 28 cases.CONCLUSION: We used shape contexts method that is simple and easy to implement to achieve symmetric boundaries for left and right breasts boundaries in thermal images.

Incoherent digital holographic spectral imaging with high accuracy of image pixel registration

Fresnel incoherent correlation holography(FINCH) is a unique three-dimensional(3D) imaging technique which has the advantages of scanning-free,high resolution,and easy matching with existing mature optical systems.In this article,an incoherent digital holographic spectral imaging method with high accuracy of spectral reconstruction based on liquid crystal tunable filter(LCTF) and FINCH is proposed.Using the programmable characteristics of spatial light modulator(SLM),a series of phase masks,none of whose focal lengths changes with wavelength,is designed and made.For each wavelength of LCTF output,SLM calls three phase masks with different phase constants at the corresponding wavelength,and CCD records three holograms.The spectral images obtained by this method have a constant magnification,which can achieve pixel-level image registration,restrain image registration errors,and improve spectral reconstruction accuracy.The results show that this method can not only obtain the 3D spatial information and spectral information of the object simultaneously,but also have high accuracy of spectral reconstruction and excellent color reproducibility.

A NEW IMAGE REGISTRATION METHOD FOR GREY IMAGES

The proposed algorithm relies on a group of new formulas for calculating tangent slope so as to address angle feature of edge curves of image. It can utilize tangent angle features to estimate automatically and fully the rotation parameters of geometric transform and enable rough matching of images with huge rotation difference. After angle compensation, it can search for matching point sets by correlation criterion, then calculate parameters of affine transform, enable higher-precision emendation of rotation and transferring. Finally, it fulfills precise matching for images with relax-tense iteration method. Compared with the registration approach based on wavelet direction-angle features, the matching algorithm with tangent feature of image edge is more robust and realizes precise registration of various images. Furthermore, it is also helpful in graphics matching.

THE STUDY ON AUTOMATIC REGISTRATION OF INSAR IMAGES

Interferometric Synthetic Aperture Radar (InSAR) allows production of high resolution DEM and detection of small earth motions using multiple pass SAR data sets obtained by remote sensing satellite. But the whole process has not yet reached sufficient robustness to warrant automated DEM production as commonly produced by stereovision with optical images. The automatic algorithm for precision registration is one of the bottlenecks in InSAR data processing. In this paper, an automatic approach with multi-step image matching algorithm is presented. All procedures are automatically implemented. The experiment is carried out successfully with SIR-C/L-band InSAR data. The triangular piecewise rectification is also advanced in reducing local distortion between the images and processing the large scene image. The primary result has prospect in the precision registration for the repeat-track InSAR data and reveals the potential of the presented automatic strategy.

Total Variation Constrained Non-Negative Matrix Factorization for Medical Image Registration

This paper presents a novel medical image registration algorithm named total variation constrained graphregularization for non-negative matrix factorization(TV-GNMF).The method utilizes non-negative matrix factorization by total variation constraint and graph regularization.The main contributions of our work are the following.First,total variation is incorporated into NMF to control the diffusion speed.The purpose is to denoise in smooth regions and preserve features or details of the data in edge regions by using a diffusion coefficient based on gradient information.Second,we add graph regularization into NMF to reveal intrinsic geometry and structure information of features to enhance the discrimination power.Third,the multiplicative update rules and proof of convergence of the TV-GNMF algorithm are given.Experiments conducted on datasets show that the proposed TV-GNMF method outperforms other state-of-the-art algorithms.

Multi-sensor image registration using multi-resolution shape analysis

Multi-sensor image registration has been widely used in remote sensing and medical image field, but registration performance is degenerated when heterogeneous images are involved. An image registration method based on multi-resolution shape analysis is proposed in this paper, to deal with the problem that the shape of similar objects is always invariant. The contours of shapes are first detected as visual features using an extended contour search algorithm in order to reduce effects of noise, and the multi-resolution shape descriptor is constructed through Fourier curvature representation of the contour’s chain code. Then a minimum distance function is used to judge the similarity between two contours. To avoid the effect of different resolution and intensity distribution, suitable resolution of each image is selected by maximizing the consistency of its pyramid shapes. Finally, the transformation parameters are estimated based on the matched control-point pairs which are the centers of gravity of the closed contours. Multi-sensor Landsat TM imagery and infrared imagery have been used as experimental data for comparison with the classical contour-based registration. Our results have been shown to be superior to the classical ones.

Image registration of the human accommodating eye demonstrates equivalent increases in lens equatorial radius and central thickness

AIM: To compare the results of in vivo human high resolution image registration studies of the eye during accommodation to the predictions of mathematical and finite element models of accommodation. METHODS: Data from published high quality image registration studies of pilocarpine induced accommodative changes of equatorial lens radius(ELR) and central lens thickness(CLT) were statistically analyzed. RESULTS: The mean changes in ELR and CLT were 6.76 μm/diopter and 6.51 μm/diopter, respectively. The linear regressions, reflecting the association between ELR and accommodative amplitude(AAELR) was: slope=6.58 μm/diopter, r^2=0.98, P<0.0001 and between CLT and AACLT was: slope=6.75 μm/diopter, r^2=0.83, P<0.001. On the basis of these relationships, the CLT slope and the AAELR were used to predict the measured change in ELR(ELRpredicted). There was no statistical difference between ELRpredicted and the measured ELR as demonstrated by a Student's paired t-test: P=0.96 and linear regression analysis: slope=0.97, r^2=0.98, P<0.00001.CONCLUSION: Image registration with invariant positional references demonstrates that ELR and CLT equivalently minimally increase ~7.0 μm/diopter during accommodation. The small equivalent increases in ELRand CLT are associated with a large accommodative amplitude. These findings are consistent with the predictions of mathematical and finite element models that specified the stiffness of the lens nucleus is the same or greater than the lens cortex and that accommodation involves a small force(<5 g).

Optical and SAR image registration based on improved nonsubsampled wavelet transform for sea islands

Homologous feature point extraction is a key problem in the optical and synthetic aperture radar （SAR） image registration for islands. A new feature point extraction method using a threshold shrink operator and non-subsampled wavelet transform （TSO-NSWT） for optical and SAR image registration was proposed. Moreover, the matching for this proposed feature was different from the traditional feature matching strategies and was performed using a similarity measure computed from neighborhood circles in low-frequency bands. Then, a number of reliably matched couples with even distributions were obtained, which assured the accuracy of the registration. Application of the proposed algorithm to SPOT-5 （multi-spectral） and YG-1 （SAR） images showed that a large number of accurately matched couples could be identified. Additionally, both of the root mean square error （RMSE） patterns of the registration parameters computed based on the TSO-NSWT algorithm and traditional NSWT algorithm were analyzed and compared, which further demonstrated the effectiveness of the proposed algorithm. The algorithm can supply the crucial step for island image registration and island recognition.

High-Dimensional Spatial Standardization Algorithm for Diffusion Tensor Image Registration

Three high dimensional spatial standardization algorithms are used for diffusion tensor image(DTI)registration,and seven kinds of methods are used to evaluate their performances.Firstly,the template used in this paper was obtained by spatial transformation of 16 subjects by means of tensor-based standardization.Then,high dimensional standardization algorithms for diffusion tensor images,including fractional anisotropy(FA)based diffeomorphic registration algorithm,FA based elastic registration algorithm and tensor-based registration algorithm,were performed.Finally,7 kinds of evaluation methods,including normalized standard deviation,dyadic coherence,diffusion cross-correlation,overlap of eigenvalue-eigenvector pairs,Euclidean distance of diffusion tensor,and Euclidean distance of the deviatoric tensor and deviatoric of tensors,were used to qualitatively compare and summarize the above standardization algorithms.Experimental results revealed that the high-dimensional tensor-based standardization algorithms perform well and can maintain the consistency of anatomical structures.

A Mosaic Technique Based on Image Registration

osaic technique and image registration process are studied in this paper. By image registration, the correspondence between images of the same scene can be established. The aim of mosaic is to build up the scene panorama of a real image sequence of the same scene. So mosaic can be integrated by way of image registration. Here, some image registration method is given, and then affine transformation is taken. Finally, the image sequence is built into the panorama.

Feasibility Study on Deformable Image Registration for Lung SBRT Patients for Dose-Driven Adaptive Therapy

The purpose of the study was to evaluate a treatment dose using planning computed tomography (pCT) that was deformed to pre-treatment cone beam computed tomography (CBCT) for lung stereotactic body radiation therapy (SBRT) treatment. Five lung SBRT patients were retrospectively selected, and their daily CBCTs were employed in this study. Dosimetric comparison was performed between the original and recalculated plans from the deformed pCT (dose per fraction) by comparing a target coverage and organs at risk. Dose summation of five fractions was computed and compared to the original plan. A phantom study was conducted to evaluate the dosimetric accuracy for the dose per fraction. In the phantom study, the difference between the mean Hounsfield Unit (HU) values of the original and deformed pCTs is less than 0.5%. In patient study, the mean HU deviation of the five deformed pCTs compared to that of the original pCT was within ±5%, which is dosimetrically insignificant. While the internal target volume (ITV) shrank by 17% on average among the five patients, mean lung dose (MLD) increased by up to 7%, and D95% of PTV decreased slightly but stayed within 5%. Results showed that MLD might be a better indicative metric of normal lung dose than V20Gy as the ITV volume decreases. This study showed a feasibility to use a deformed pCT for evaluation of the dose per fraction and for a possible plan adaptation in lung SBRT cases. Readers should be cautious in selecting patients before clinical application due to the image quality of CBCT.