A method based on mutual information and gradient information for medical image registration

Mutual information is widely used in medical image registration, because it does not require preprocessing the image. However, the local maximum problem in the registration is insurmountable. We combine mutual information and gradient information to solve this problem and apply it to the non-rigid deformation image registration. To improve the accuracy, we provide some implemental issues, for example, the Powell searching algorithm, gray interpolation and consideration of outlier points. The experimental results show the accuracy of the method and the feasibility in non-rigid medical image registration.

A Review of Point Feature Based Medical Image Registration

Point features, as the basis of lines, surfaces, and bodies, are commonly used in medical image registration. To obtain an elegant spatial transformation of extracted feature points, many point set matching algorithms(PMs) have been developed to match two point sets by optimizing multifarious distance functions. There are ample reviews related to medical image registration and PMs which summarize their basic principles and main algorithms separately. However,to data, detailed summary of PMs used in medical image registration in different clinical environments has not been published. In this paper, we provide a comprehensive review of the existing key techniques of the PMs applied to medical image registration according to the basic principles and clinical applications. As the core technique of the PMs, geometric transformation models are elaborated in this paper, demonstrating the mechanism of point set registration. We also focus on the clinical applications of the PMs and propose a practical classification method according to their applications in different clinical surgeries. The aim of this paper is to provide a summary of pointfeaturebased methods used in medical image registration and to guide doctors or researchers interested in this field to choose appropriate techniques in their research.

Interpolation Algorithm Research for Medical Image Registration

Owing to its property of applying multi-modality imaging information into the clinical usage has the Medical Image Registration been the research focus. The gray nearest neighbor interpolation and bilinear interpolation and cubic convolution interpolation method used to medical image interpolation algorithm. It compares the characteristics of the three gray interpolations. Combined with the characteristics of the above algorithm is proposed based on gray-scale pixel intensity interpolation. The algorithm can be improved in the time and registration accuracy. It combined with the improved optimization algorithm in the proposed image registration of the simulation experiment, experimental precision subpixel image to verify the validity of the method.

Total Variation Constrained Non-Negative Matrix Factorization for Medical Image Registration

This paper presents a novel medical image registration algorithm named total variation constrained graphregularization for non-negative matrix factorization(TV-GNMF).The method utilizes non-negative matrix factorization by total variation constraint and graph regularization.The main contributions of our work are the following.First,total variation is incorporated into NMF to control the diffusion speed.The purpose is to denoise in smooth regions and preserve features or details of the data in edge regions by using a diffusion coefficient based on gradient information.Second,we add graph regularization into NMF to reveal intrinsic geometry and structure information of features to enhance the discrimination power.Third,the multiplicative update rules and proof of convergence of the TV-GNMF algorithm are given.Experiments conducted on datasets show that the proposed TV-GNMF method outperforms other state-of-the-art algorithms.

Medical Image Registration Based on Phase Congruency and Regional Mutual Information

In this paper, a new approach of muhi-modality image registration is represented with not only image intensity, but also features describing image structure. There are two novelties in the proposed method. Firstly, instead of standard mutual information （ MI ） based on joint intensity histogram, regional mutual information （ RMI ） is employed, which allows neighborhood information to be taken into account. Secondly, a new feature images obtained by means of phase congruency are invariants to brightness or contrast changes. By incorporating these features and intensity into RMI, we can combine the aspects of both structural and neighborhood information together, which offers a more robust and a high level of registration accuracy.

Medical Image Registration Using the Fourier Transform

A Fourier Transform (FT) based pattern-matching algorithm was adapted for use in medical image registration. This algorithm obtained the FT of two images, determined the normalized cross-power spectrum of the transformed images, and then applied an inverse FT. The result was a delta function with a maximum value at the location corresponding to the distance between the two images;a similar method was used to recover rotations. This algorithm was first tested using a simple two-dimensional image, with induced shifts of ±20 pixels and ±10 degrees. All translations were recovered with no error and all rotations were recovered within 0.18 degrees. Subsequently, this algorithm was tested on eight clinical kV images drawn from four different body sites. Twenty-five random shifts and rotations were applied to each image. The average mean error of the registration solution was -0.002 ± 0.077 mm in the x direction, 0.002 ± 0.075 mm in the y direction, and -0.012 ± 0.099 degrees. These initial results suggest that a FT algorithm has a high degree of accuracy when registering clinical kV images.

GAN-DIRNet:A Novel Deformable Image Registration Approach for Multimodal Histological Images

Multi-modal histological image registration tasks pose significant challenges due to tissue staining operations causing partial loss and folding of tissue.Convolutional neural network(CNN)and generative adversarial network(GAN)are pivotal inmedical image registration.However,existing methods often struggle with severe interference and deformation,as seen in histological images of conditions like Cushing’s disease.We argue that the failure of current approaches lies in underutilizing the feature extraction capability of the discriminator inGAN.In this study,we propose a novel multi-modal registration approach GAN-DIRNet based on GAN for deformable histological image registration.To begin with,the discriminators of two GANs are embedded as a new dual parallel feature extraction module into the unsupervised registration networks,characterized by implicitly extracting feature descriptors of specific modalities.Additionally,modal feature description layers and registration layers collaborate in unsupervised optimization,facilitating faster convergence and more precise results.Lastly,experiments and evaluations were conducted on the registration of the Mixed National Institute of Standards and Technology database(MNIST),eight publicly available datasets of histological sections and the Clustering-Registration-Classification-Segmentation(CRCS)dataset on the Cushing’s disease.Experimental results demonstrate that our proposed GAN-DIRNet method surpasses existing approaches like DIRNet in terms of both registration accuracy and time efficiency,while also exhibiting robustness across different image types.

Application of Opening and Closing Morphology in Deep Learning-Based Brain Image Registration

In order to improve the registration accuracy of brain magnetic resonance images(MRI),some deep learning registration methods use segmentation images for training model.How-ever,the segmentation values are constant for each label,which leads to the gradient variation con-centrating on the boundary.Thus,the dense deformation field(DDF)is gathered on the boundary and there even appears folding phenomenon.In order to fully leverage the label information,the morphological opening and closing information maps are introduced to enlarge the non-zero gradi-ent regions and improve the accuracy of DDF estimation.The opening information maps supervise the registration model to focus on smaller,narrow brain regions.The closing information maps supervise the registration model to pay more attention to the complex boundary region.Then,opening and closing morphology networks(OC_Net)are designed to automatically generate open-ing and closing information maps to realize the end-to-end training process.Finally,a new registra-tion architecture,VM_(seg+oc),is proposed by combining OC_Net and VoxelMorph.Experimental results show that the registration accuracy of VM_(seg+oc) is significantly improved on LPBA40 and OASIS1 datasets.Especially,VM_(seg+oc) can well improve registration accuracy in smaller brain regions and narrow regions.

Free form deformation and symmetry constraint‐based multimodal brain image registration using generative adversarial nets

Multi‐modal brain image registration has been widely applied to functional localisation,neurosurgery and computational anatomy.The existing registration methods based on the dense deformation fields involve too many parameters,which is not conducive to the exploration of correct spatial correspondence between the float and reference images.Meanwhile,the unidirectional registration may involve the deformation folding,which will result in the change of topology during registration.To address these issues,this work has presented an unsupervised image registration method using the free form deformation(FFD)and the symmetry constraint‐based generative adversarial networks(FSGAN).The FSGAN utilises the principle component analysis network‐based structural representations of the reference and float images as the inputs and uses the generator to learn the FFD model parameters,thereby producing two deformation fields.Meanwhile,the FSGAN uses two discriminators to decide whether the bilateral registration have been realised simultaneously.Besides,the symmetry constraint is utilised to construct the loss function,thereby avoiding the deformation folding.Experiments on BrainWeb,high grade gliomas,IXI and LPBA40 show that compared with state‐of‐the‐art methods,the FSGAN provides superior performance in terms of visual comparisons and such quantitative indexes as dice value,target registration error and computational efficiency.

MUTUAL INFORMATION BASED 3D NON-RIGID REGISTRATION OF CT/MR ABDOMEN IMAGES

A mutual information based 3D non-rigid registration approach was proposed for the registration of deformable CT/MR body abdomen images. The Parzen Windows Density Estimation (PWDE) method is adopted to calculate the mutual information between the two modals of CT and MRI abdomen images. By maximizing MI between the CT and MR volume images, the overlapping part of them reaches the biggest, which means that the two body images of CT and MR matches best to each other. Visible Human Project (VHP) Male abdomen CT and MRI Data are used as experimental data sets. The experimental results indicate that this approach of non-rigid 3D registration of CT/MR body abdominal images can be achieved effectively and automatically, without any prior processing procedures such as segmentation and feature extraction, but has a main drawback of very long computation time.

Non-Rigid Image Registration Algorithm Based on B-Splines Approximation

An intensity-based non-rigid registration algorithm is discussed, which uses Gaussian smoothing to constrain the transformation to be smooth, and thus preserves the topology of images. In view of the insufficiency of the uniform Gaussian filtering of the deformation field, an automatic and accurate non-rigid image registration method based on B-splines approximation is proposed. The regularization strategy is adopted by using multi-level B-splines approximation to regularize the displacement fields in a coarse-to-fine manner. Moreover, it assigns the different weights to the estimated displacements according to their reliabilities. In this way, the level of regularity can be adapted locally. Experiments were performed on both synthetic and real medical images of brain, and the results show that the proposed method improves the registration accuracy and robustness.

Fast Mutual Information Registration Method of 3-D Medical Image

Currently the voxel based registration methods have been used widely such as the well known mutual information (MI). Although the accuracy of their results is plausible, the registration procedure is slow. This paper proposed some methods to rigid registration based on mutual information, aiming for an acceleration of the registration process without significantly loss of precision in the final results. The efficiency of these methods is examined by registration of CT MR and PET MR. Experimental results show that the speedup is effective and efficient. By using the fast methods, the registration of 3 D medical image could also be implemented on PC rapidly.

SuperFusion: A Versatile Image Registration and Fusion Network with Semantic Awareness

Image fusion aims to integrate complementary information in source images to synthesize a fused image comprehensively characterizing the imaging scene. However, existing image fusion algorithms are only applicable to strictly aligned source images and cause severe artifacts in the fusion results when input images have slight shifts or deformations. In addition,the fusion results typically only have good visual effect, but neglect the semantic requirements of high-level vision tasks.This study incorporates image registration, image fusion, and semantic requirements of high-level vision tasks into a single framework and proposes a novel image registration and fusion method, named Super Fusion. Specifically, we design a registration network to estimate bidirectional deformation fields to rectify geometric distortions of input images under the supervision of both photometric and end-point constraints. The registration and fusion are combined in a symmetric scheme, in which while mutual promotion can be achieved by optimizing the naive fusion loss, it is further enhanced by the mono-modal consistent constraint on symmetric fusion outputs. In addition, the image fusion network is equipped with the global spatial attention mechanism to achieve adaptive feature integration. Moreover, the semantic constraint based on the pre-trained segmentation model and Lovasz-Softmax loss is deployed to guide the fusion network to focus more on the semantic requirements of high-level vision tasks. Extensive experiments on image registration, image fusion,and semantic segmentation tasks demonstrate the superiority of our Super Fusion compared to the state-of-the-art alternatives.The source code and pre-trained model are publicly available at https://github.com/Linfeng-Tang/Super Fusion.

A NEW IMAGE REGISTRATION METHOD FOR GREY IMAGES

The proposed algorithm relies on a group of new formulas for calculating tangent slope so as to address angle feature of edge curves of image. It can utilize tangent angle features to estimate automatically and fully the rotation parameters of geometric transform and enable rough matching of images with huge rotation difference. After angle compensation, it can search for matching point sets by correlation criterion, then calculate parameters of affine transform, enable higher-precision emendation of rotation and transferring. Finally, it fulfills precise matching for images with relax-tense iteration method. Compared with the registration approach based on wavelet direction-angle features, the matching algorithm with tangent feature of image edge is more robust and realizes precise registration of various images. Furthermore, it is also helpful in graphics matching.

Multi-sensor image registration using multi-resolution shape analysis

Multi-sensor image registration has been widely used in remote sensing and medical image field, but registration performance is degenerated when heterogeneous images are involved. An image registration method based on multi-resolution shape analysis is proposed in this paper, to deal with the problem that the shape of similar objects is always invariant. The contours of shapes are first detected as visual features using an extended contour search algorithm in order to reduce effects of noise, and the multi-resolution shape descriptor is constructed through Fourier curvature representation of the contour’s chain code. Then a minimum distance function is used to judge the similarity between two contours. To avoid the effect of different resolution and intensity distribution, suitable resolution of each image is selected by maximizing the consistency of its pyramid shapes. Finally, the transformation parameters are estimated based on the matched control-point pairs which are the centers of gravity of the closed contours. Multi-sensor Landsat TM imagery and infrared imagery have been used as experimental data for comparison with the classical contour-based registration. Our results have been shown to be superior to the classical ones.

Optical and SAR image registration based on improved nonsubsampled wavelet transform for sea islands

Homologous feature point extraction is a key problem in the optical and synthetic aperture radar （SAR） image registration for islands. A new feature point extraction method using a threshold shrink operator and non-subsampled wavelet transform （TSO-NSWT） for optical and SAR image registration was proposed. Moreover, the matching for this proposed feature was different from the traditional feature matching strategies and was performed using a similarity measure computed from neighborhood circles in low-frequency bands. Then, a number of reliably matched couples with even distributions were obtained, which assured the accuracy of the registration. Application of the proposed algorithm to SPOT-5 （multi-spectral） and YG-1 （SAR） images showed that a large number of accurately matched couples could be identified. Additionally, both of the root mean square error （RMSE） patterns of the registration parameters computed based on the TSO-NSWT algorithm and traditional NSWT algorithm were analyzed and compared, which further demonstrated the effectiveness of the proposed algorithm. The algorithm can supply the crucial step for island image registration and island recognition.

Multi-modality liver image registration based on multilevel B-splines free-form deformation and L-BFGS optimal algorithm

A new coarse-to-fine strategy was proposed for nonrigid registration of computed tomography(CT) and magnetic resonance(MR) images of a liver.This hierarchical framework consisted of an affine transformation and a B-splines free-form deformation(FFD).The affine transformation performed a rough registration targeting the mismatch between the CT and MR images.The B-splines FFD transformation performed a finer registration by correcting local motion deformation.In the registration algorithm,the normalized mutual information(NMI) was used as similarity measure,and the limited memory Broyden-Fletcher- Goldfarb-Shannon(L-BFGS) optimization method was applied for optimization process.The algorithm was applied to the fully automated registration of liver CT and MR images in three subjects.The results demonstrate that the proposed method not only significantly improves the registration accuracy but also reduces the running time,which is effective and efficient for nonrigid registration.

Two fully automated data-driven 3D whole-breast segmentation strategies in MRI for MR-based breast density using image registration and U-Net with a focus on reproducibility

Presence of higher breast density(BD)and persistence over time are risk factors for breast cancer.A quantitatively accurate and highly reproducible BD measure that relies on precise and reproducible whole-breast segmentation is desirable.In this study,we aimed to develop a highly reproducible and accurate whole-breast segmentation algorithm for the generation of reproducible BD measures.Three datasets of volunteers from two clinical trials were included.Breast MR images were acquired on 3T Siemens Biograph mMR,Prisma,and Skyra using 3D Cartesian six-echo GRE sequences with a fat-water separation technique.Two whole-breast segmentation strategies,utiliz-ing image registration and 3D U-Net,were developed.Manual segmentation was performed.A task-based analysis was performed:a previously developed MR-based BD measure,MagDensity,was calculated and assessed using automated and manual segmentation.The mean squared error(MSE)and intraclass correlation coefficient(ICC)between MagDensity were evaluated using the manual segmentation as a reference.The test-retest reproducibility of MagDensity derived from different breast segmentation methods was assessed using the difference between the test and retest measures(Δ_(2-1)),MSE,and ICC.The results showed that MagDensity derived by the registration and deep learning segmentation methods exhibited high concordance with manual segmentation,with ICCs of 0.986(95%CI:0.974-0.993)and 0.983(95%CI:0.961-0.992),respectively.For test-retest analysis,MagDensity derived using the regis-tration algorithm achieved the smallest MSE of 0.370 and highest ICC of 0.993(95%CI:0.982-0.997)when compared to other segmentation methods.In conclusion,the proposed registration and deep learning whole-breast segmentation methods are accurate and reliable for estimating BD.Both methods outperformed a previously developed algorithm and manual segmentation in the test-retest assessment,with the registration exhibiting superior performance for highly reproducible BD measurements.

Sonar Image Registration and Mosaic Based on Line Detection and Triangle Matching

Image registration is an important research topic in the field of computer vision,in which the registration and mosaic of side-scan sonar images is the keypoints of underwater navigation.However,the image registration method of keypoints is not suitable for sonar images which do not have obvious feature points.Therefore,a method of sonar-image registration and mosaic based on line segment extraction and triangle matching is proposed in this paper.Firstly,in order to extract features from sonar image,the LSD method is introduced to detect line feature from images,and line segments are filtered by the principle of attention;after that,triangles are formed from line segments,an image transformation matrix can be calculated through the heuristic greedy algorithm from these triangles;finally,images are merged based on the transformation information.On the basis of practical tests,it is found that,the feature extraction method used in this paper can better describe the outline of underwater terrain,and there is no obvious stitching gap between the result of sonar images stitched.Experimental results show that the proposed method is effective than the keypoints method of the registration and mosaic of sonar images.

Image Registration Based on Improved Mutual Information with Hybrid Optimizer

An improved image registration method is proposed based on mutual infor- mation with hybrid optimizer. Firstly, mutual information measure is combined with morphological gradient information. The essence of the gradient information is that locations a large gradient magnitude should be aligned, but also the orientation of the gradients at those locations should be similar. Secondly, a hybrid optimizer combined PSO with Powell algorithm is proposed to restrain local maxima of mutual information function and improve the registration accuracy to sub-pixel level. Lastly, muhlresolution data structure based on Mallat decomposition can not only improve the behavior of registration function, but also improve the speed of the algorithm. Experimental results demonstrate that the new method can yield good registration result, superior to traditional optimizer with respect to smoothness and attraction basin as well as convergence speed.