Non-Rigid Image Registration Algorithm Based on B-Splines Approximation

An intensity-based non-rigid registration algorithm is discussed, which uses Gaussian smoothing to constrain the transformation to be smooth, and thus preserves the topology of images. In view of the insufficiency of the uniform Gaussian filtering of the deformation field, an automatic and accurate non-rigid image registration method based on B-splines approximation is proposed. The regularization strategy is adopted by using multi-level B-splines approximation to regularize the displacement fields in a coarse-to-fine manner. Moreover, it assigns the different weights to the estimated displacements according to their reliabilities. In this way, the level of regularity can be adapted locally. Experiments were performed on both synthetic and real medical images of brain, and the results show that the proposed method improves the registration accuracy and robustness.

Nonlinear Registration of Brain Magnetic Resonance Images with Cross Constraints of Intensity and Structure

Many deep learning-based registration methods rely on a single-stream encoder-decoder network for computing deformation fields between 3D volumes.However,these methods often lack constraint information and overlook semantic consistency,limiting their performance.To address these issues,we present a novel approach for medical image registration called theDual-VoxelMorph,featuring a dual-channel cross-constraint network.This innovative network utilizes both intensity and segmentation images,which share identical semantic information and feature representations.Two encoder-decoder structures calculate deformation fields for intensity and segmentation images,as generated by the dual-channel cross-constraint network.This design facilitates bidirectional communication between grayscale and segmentation information,enabling the model to better learn the corresponding grayscale and segmentation details of the same anatomical structures.To ensure semantic and directional consistency,we introduce constraints and apply the cosine similarity function to enhance semantic consistency.Evaluation on four public datasets demonstrates superior performance compared to the baselinemethod,achieving Dice scores of 79.9%,64.5%,69.9%,and 63.5%for OASIS-1,OASIS-3,LPBA40,and ADNI,respectively.

Real-time continuous image guidance for endoscopic retrograde cholangiopancreatography based on 3D/2D registration and respiratory compensation

BACKGROUND It has been confirmed that three-dimensional(3D)imaging allows easier identification of bile duct anatomy and intraoperative guidance of endoscopic retrograde cholangiopancreatography(ERCP),which reduces the radiation dose and procedure time with improved safety.However,current 3D biliary imaging does not have good real-time fusion with intraoperative imaging,a process meant to overcome the influence of intraoperative respiratory motion and guide navigation.The present study explored the feasibility of real-time continuous image-guided ERCP.AIM To explore the feasibility of real-time continuous image-guided ERCP.METHODS We selected 23D-printed abdominal biliary tract models with different structures to simulate different patients.The ERCP environment was simulated for the biliary phantom experiment to create a navigation system,which was further tested in patients.In addition,based on the estimation of the patient’s respiratory motion,preoperative 3D biliary imaging from computed tomography of 18 patients with cholelithiasis was registered and fused in real-time with 2D fluoroscopic sequence generated by the C-arm unit during ERCP.RESULTS Continuous image-guided ERCP was applied in the biliary phantom with a registration error of 0.46 mm±0.13 mm and a tracking error of 0.64 mm±0.24mm.After estimating the respiratory motion,3D/2D registration accurately transformed preoperative 3D biliary images to each image in the X-ray image sequence in real-time in 18 patients,with an average fusion rate of 88%.CONCLUSION Continuous image-guided ERCP may be an effective approach to assist the operator and reduce the use of X-ray and contrast agents.

Free form deformation and symmetry constraint‐based multimodal brain image registration using generative adversarial nets

Multi‐modal brain image registration has been widely applied to functional localisation,neurosurgery and computational anatomy.The existing registration methods based on the dense deformation fields involve too many parameters,which is not conducive to the exploration of correct spatial correspondence between the float and reference images.Meanwhile,the unidirectional registration may involve the deformation folding,which will result in the change of topology during registration.To address these issues,this work has presented an unsupervised image registration method using the free form deformation(FFD)and the symmetry constraint‐based generative adversarial networks(FSGAN).The FSGAN utilises the principle component analysis network‐based structural representations of the reference and float images as the inputs and uses the generator to learn the FFD model parameters,thereby producing two deformation fields.Meanwhile,the FSGAN uses two discriminators to decide whether the bilateral registration have been realised simultaneously.Besides,the symmetry constraint is utilised to construct the loss function,thereby avoiding the deformation folding.Experiments on BrainWeb,high grade gliomas,IXI and LPBA40 show that compared with state‐of‐the‐art methods,the FSGAN provides superior performance in terms of visual comparisons and such quantitative indexes as dice value,target registration error and computational efficiency.

Application of Opening and Closing Morphology in Deep Learning-Based Brain Image Registration

In order to improve the registration accuracy of brain magnetic resonance images(MRI),some deep learning registration methods use segmentation images for training model.How-ever,the segmentation values are constant for each label,which leads to the gradient variation con-centrating on the boundary.Thus,the dense deformation field(DDF)is gathered on the boundary and there even appears folding phenomenon.In order to fully leverage the label information,the morphological opening and closing information maps are introduced to enlarge the non-zero gradi-ent regions and improve the accuracy of DDF estimation.The opening information maps supervise the registration model to focus on smaller,narrow brain regions.The closing information maps supervise the registration model to pay more attention to the complex boundary region.Then,opening and closing morphology networks(OC_Net)are designed to automatically generate open-ing and closing information maps to realize the end-to-end training process.Finally,a new registra-tion architecture,VM_(seg+oc),is proposed by combining OC_Net and VoxelMorph.Experimental results show that the registration accuracy of VM_(seg+oc) is significantly improved on LPBA40 and OASIS1 datasets.Especially,VM_(seg+oc) can well improve registration accuracy in smaller brain regions and narrow regions.

MUTUAL INFORMATION BASED 3D NON-RIGID REGISTRATION OF CT/MR ABDOMEN IMAGES

A mutual information based 3D non-rigid registration approach was proposed for the registration of deformable CT/MR body abdomen images. The Parzen Windows Density Estimation (PWDE) method is adopted to calculate the mutual information between the two modals of CT and MRI abdomen images. By maximizing MI between the CT and MR volume images, the overlapping part of them reaches the biggest, which means that the two body images of CT and MR matches best to each other. Visible Human Project (VHP) Male abdomen CT and MRI Data are used as experimental data sets. The experimental results indicate that this approach of non-rigid 3D registration of CT/MR body abdominal images can be achieved effectively and automatically, without any prior processing procedures such as segmentation and feature extraction, but has a main drawback of very long computation time.

Non-rigid registration of medical images based on S^(1/2)(Δ_(mn)^((2)))non-tensor product Bspline

In this study,a non-tensor product B-spline algorithm is applied to the search space of the registration process,and a new method of image non-rigid registration is proposed.The tensor product B-spline is a function defined in the two directions of x and y,while the non-tensor product B-spline S^(1/2)(Δ_(mn)^((2)))is defined in four directions on the 2-type triangulation.For certain problems,using non-tensor product B-splines to describe the non-rigid deformation of an image can more accurately extract the four-directional information of the image,thereby describing the global or local non-rigid deformation of the image in more directions.Indeed,it provides a method to solve the problem of image deformation in multiple directions.In addition,the region of interest of medical images is irregular,and usually no value exists on the boundary triangle.The value of the basis function of the non-tensor product B-spline on the boundary triangle is only 0.The algorithm process is optimized.The algorithm performs completely automatic non-rigid registration of computed tomography and magnetic resonance imaging images of patients.In particular,this study compares the performance of the proposed algorithm with the tensor product B-spline registration algorithm.The results elucidate that the proposed algorithm clearly improves the accuracy.

Research on Detection Technology of Micro-Components on Circuit Board Based on Digital Image Processing

Aiming at the stability of the circuit board image in the acquisition process,this paper realizes the accurate registration of the image to be registered and the standard image based on the SIFT feature operator and RANSAC algorithm.The device detection model and data set are established based on Faster RCNN.Finally,the number of training was continuously optimized,and when the loss function of Faster RCNN converged,the identification result of the device was obtained.

Automatic Extraction of the Sparse Prior Correspondences for Non-Rigid Point Cloud Registration

Non-rigid registration of point clouds is still far from stable,especially for the largely deformed one.Sparse initial correspondences are often adopted to facilitate the process.However,there are few studies on how to build them automatically.Therefore,in this paper,we propose a robust method to compute such priors automatically,where a global and local combined strategy is adopted.These priors in different degrees of deformation are obtained by the locally geometrical-consistent point matches from the globally structural-consistent region correspondences.To further utilize the matches,this paper also proposes a novel registration method based on the Coherent Point Drift framework.This method takes both the spatial proximity and local structural consistency of the priors as supervision of the registration process and thus obtains a robust alignment for clouds with significantly different deformations.Qualitative and quantitative experiments demonstrate the advantages of the proposed method.

A Review of Point Feature Based Medical Image Registration

Point features, as the basis of lines, surfaces, and bodies, are commonly used in medical image registration. To obtain an elegant spatial transformation of extracted feature points, many point set matching algorithms(PMs) have been developed to match two point sets by optimizing multifarious distance functions. There are ample reviews related to medical image registration and PMs which summarize their basic principles and main algorithms separately. However,to data, detailed summary of PMs used in medical image registration in different clinical environments has not been published. In this paper, we provide a comprehensive review of the existing key techniques of the PMs applied to medical image registration according to the basic principles and clinical applications. As the core technique of the PMs, geometric transformation models are elaborated in this paper, demonstrating the mechanism of point set registration. We also focus on the clinical applications of the PMs and propose a practical classification method according to their applications in different clinical surgeries. The aim of this paper is to provide a summary of pointfeaturebased methods used in medical image registration and to guide doctors or researchers interested in this field to choose appropriate techniques in their research.

SuperFusion: A Versatile Image Registration and Fusion Network with Semantic Awareness

Image fusion aims to integrate complementary information in source images to synthesize a fused image comprehensively characterizing the imaging scene. However, existing image fusion algorithms are only applicable to strictly aligned source images and cause severe artifacts in the fusion results when input images have slight shifts or deformations. In addition,the fusion results typically only have good visual effect, but neglect the semantic requirements of high-level vision tasks.This study incorporates image registration, image fusion, and semantic requirements of high-level vision tasks into a single framework and proposes a novel image registration and fusion method, named Super Fusion. Specifically, we design a registration network to estimate bidirectional deformation fields to rectify geometric distortions of input images under the supervision of both photometric and end-point constraints. The registration and fusion are combined in a symmetric scheme, in which while mutual promotion can be achieved by optimizing the naive fusion loss, it is further enhanced by the mono-modal consistent constraint on symmetric fusion outputs. In addition, the image fusion network is equipped with the global spatial attention mechanism to achieve adaptive feature integration. Moreover, the semantic constraint based on the pre-trained segmentation model and Lovasz-Softmax loss is deployed to guide the fusion network to focus more on the semantic requirements of high-level vision tasks. Extensive experiments on image registration, image fusion,and semantic segmentation tasks demonstrate the superiority of our Super Fusion compared to the state-of-the-art alternatives.The source code and pre-trained model are publicly available at https://github.com/Linfeng-Tang/Super Fusion.

Using shape contexts method for registration of contra lateral breasts in thermal images

AIM: To achieve symmetric boundaries for left and right breasts boundaries in thermal images by registration. METHODS: The proposed method for registration consists of two steps. In the first step, shape context, an approach as presented by Belongie and Malik was applied for registration of two breast boundaries. The shape context is an approach to measure shape similarity. Two sets of finite sample points from shape contours of two breasts are then presented. Consequently, the correspondences between the two shapes are found. By finding correspondences, the sample point which has the most similar shape context is obtained. RESULTS: In this study, a line up transformation which maps one shape onto the other has been estimated in order to complete shape. The used of a thin plate spline permitted good estimation of a plane transformation which has capability to map unselective points from one shape onto the other. The obtained aligningtransformation of boundaries points has been applied successfully to map the two breasts interior points. Some of advantages for using shape context method in this work are as follows:(1) no special land marks or key points are needed;(2) it is tolerant to all common shape deformation; and(3) although it is uncomplicated and straightforward to use, it gives remarkably powerful descriptor for point sets significantly upgrading point set registration. Results are very promising. The proposed algorithm was implemented for 32 cases. Boundary registration is done perfectly for 28 cases.CONCLUSION: We used shape contexts method that is simple and easy to implement to achieve symmetric boundaries for left and right breasts boundaries in thermal images.

Incoherent digital holographic spectral imaging with high accuracy of image pixel registration

Fresnel incoherent correlation holography(FINCH) is a unique three-dimensional(3D) imaging technique which has the advantages of scanning-free,high resolution,and easy matching with existing mature optical systems.In this article,an incoherent digital holographic spectral imaging method with high accuracy of spectral reconstruction based on liquid crystal tunable filter(LCTF) and FINCH is proposed.Using the programmable characteristics of spatial light modulator(SLM),a series of phase masks,none of whose focal lengths changes with wavelength,is designed and made.For each wavelength of LCTF output,SLM calls three phase masks with different phase constants at the corresponding wavelength,and CCD records three holograms.The spectral images obtained by this method have a constant magnification,which can achieve pixel-level image registration,restrain image registration errors,and improve spectral reconstruction accuracy.The results show that this method can not only obtain the 3D spatial information and spectral information of the object simultaneously,but also have high accuracy of spectral reconstruction and excellent color reproducibility.

Total Variation Constrained Non-Negative Matrix Factorization for Medical Image Registration

This paper presents a novel medical image registration algorithm named total variation constrained graphregularization for non-negative matrix factorization(TV-GNMF).The method utilizes non-negative matrix factorization by total variation constraint and graph regularization.The main contributions of our work are the following.First,total variation is incorporated into NMF to control the diffusion speed.The purpose is to denoise in smooth regions and preserve features or details of the data in edge regions by using a diffusion coefficient based on gradient information.Second,we add graph regularization into NMF to reveal intrinsic geometry and structure information of features to enhance the discrimination power.Third,the multiplicative update rules and proof of convergence of the TV-GNMF algorithm are given.Experiments conducted on datasets show that the proposed TV-GNMF method outperforms other state-of-the-art algorithms.

THE STUDY ON AUTOMATIC REGISTRATION OF INSAR IMAGES

Interferometric Synthetic Aperture Radar (InSAR) allows production of high resolution DEM and detection of small earth motions using multiple pass SAR data sets obtained by remote sensing satellite. But the whole process has not yet reached sufficient robustness to warrant automated DEM production as commonly produced by stereovision with optical images. The automatic algorithm for precision registration is one of the bottlenecks in InSAR data processing. In this paper, an automatic approach with multi-step image matching algorithm is presented. All procedures are automatically implemented. The experiment is carried out successfully with SIR-C/L-band InSAR data. The triangular piecewise rectification is also advanced in reducing local distortion between the images and processing the large scene image. The primary result has prospect in the precision registration for the repeat-track InSAR data and reveals the potential of the presented automatic strategy.

Image registration of the human accommodating eye demonstrates equivalent increases in lens equatorial radius and central thickness

AIM: To compare the results of in vivo human high resolution image registration studies of the eye during accommodation to the predictions of mathematical and finite element models of accommodation. METHODS: Data from published high quality image registration studies of pilocarpine induced accommodative changes of equatorial lens radius(ELR) and central lens thickness(CLT) were statistically analyzed. RESULTS: The mean changes in ELR and CLT were 6.76 μm/diopter and 6.51 μm/diopter, respectively. The linear regressions, reflecting the association between ELR and accommodative amplitude(AAELR) was: slope=6.58 μm/diopter, r^2=0.98, P<0.0001 and between CLT and AACLT was: slope=6.75 μm/diopter, r^2=0.83, P<0.001. On the basis of these relationships, the CLT slope and the AAELR were used to predict the measured change in ELR(ELRpredicted). There was no statistical difference between ELRpredicted and the measured ELR as demonstrated by a Student's paired t-test: P=0.96 and linear regression analysis: slope=0.97, r^2=0.98, P<0.00001.CONCLUSION: Image registration with invariant positional references demonstrates that ELR and CLT equivalently minimally increase ~7.0 μm/diopter during accommodation. The small equivalent increases in ELRand CLT are associated with a large accommodative amplitude. These findings are consistent with the predictions of mathematical and finite element models that specified the stiffness of the lens nucleus is the same or greater than the lens cortex and that accommodation involves a small force(<5 g).

High-Dimensional Spatial Standardization Algorithm for Diffusion Tensor Image Registration

Three high dimensional spatial standardization algorithms are used for diffusion tensor image(DTI)registration,and seven kinds of methods are used to evaluate their performances.Firstly,the template used in this paper was obtained by spatial transformation of 16 subjects by means of tensor-based standardization.Then,high dimensional standardization algorithms for diffusion tensor images,including fractional anisotropy(FA)based diffeomorphic registration algorithm,FA based elastic registration algorithm and tensor-based registration algorithm,were performed.Finally,7 kinds of evaluation methods,including normalized standard deviation,dyadic coherence,diffusion cross-correlation,overlap of eigenvalue-eigenvector pairs,Euclidean distance of diffusion tensor,and Euclidean distance of the deviatoric tensor and deviatoric of tensors,were used to qualitatively compare and summarize the above standardization algorithms.Experimental results revealed that the high-dimensional tensor-based standardization algorithms perform well and can maintain the consistency of anatomical structures.

Optical and SAR image registration based on improved nonsubsampled wavelet transform for sea islands

Homologous feature point extraction is a key problem in the optical and synthetic aperture radar （SAR） image registration for islands. A new feature point extraction method using a threshold shrink operator and non-subsampled wavelet transform （TSO-NSWT） for optical and SAR image registration was proposed. Moreover, the matching for this proposed feature was different from the traditional feature matching strategies and was performed using a similarity measure computed from neighborhood circles in low-frequency bands. Then, a number of reliably matched couples with even distributions were obtained, which assured the accuracy of the registration. Application of the proposed algorithm to SPOT-5 （multi-spectral） and YG-1 （SAR） images showed that a large number of accurately matched couples could be identified. Additionally, both of the root mean square error （RMSE） patterns of the registration parameters computed based on the TSO-NSWT algorithm and traditional NSWT algorithm were analyzed and compared, which further demonstrated the effectiveness of the proposed algorithm. The algorithm can supply the crucial step for island image registration and island recognition.

A Mosaic Technique Based on Image Registration

ＡＭｏｓａｉｃＴｅｃｈｎｉｑｕｅＢａｓｅｄｏｎＩｍａｇｅＲｅｇｉｓｔｒａｔｉｏｎＨｅＱｉａｎｇ（何强），ＴｉａｎＹｉｎｇｌｉ，ＭａＳｏｎｇｄｅ（ＮａｔｉｏｎａｌＬａｂｏｆＰａｔｅｒｎＲｅｃｏｇｎｉｔｉｏｎＩｎｓｔｉｔｕｔｅｏｆＡｕｔｏｍａｔｉｏｎ，...