Disordered Multi-view Registration Method Based on the Soft Trimmed Deep Network

Compared with the pair-wise registration of point clouds,multi-view point cloud registration is much less studied.In this dissertation,a disordered multi-view point cloud registration method based on the soft trimmed deep network is proposed.In this method,firstly,the expression ability of feature extraction module is improved and the registration accuracy is increased by enhancing feature extraction network with the point pair feature.Secondly,neighborhood and angle similarities are used to measure the consistency of candidate points to surrounding neighborhoods.By combining distance consistency and high dimensional feature consistency,our network introduces the confidence estimation module of registration,so the point cloud trimmed problem can be converted to candidate for the degree of confidence estimation problem,achieving the pair-wise registration of partially overlapping point clouds.Thirdly,the results from pair-wise registration are fed into the model fusion to achieve the rough registration of multi-view point clouds.Finally,the hierarchical clustering is used to iteratively optimize the clustering center model by gradually increasing the number of clustering categories and performing clustering and registration alternately.This method achieves rough point cloud registration quickly in the early stage,improves the accuracy of multi-view point cloud registration in the later stage,and makes full use of global information to achieve robust and accurate multi-view registration without initial value.

Real-time continuous image guidance for endoscopic retrograde cholangiopancreatography based on 3D/2D registration and respiratory compensation

BACKGROUND It has been confirmed that three-dimensional(3D)imaging allows easier identification of bile duct anatomy and intraoperative guidance of endoscopic retrograde cholangiopancreatography(ERCP),which reduces the radiation dose and procedure time with improved safety.However,current 3D biliary imaging does not have good real-time fusion with intraoperative imaging,a process meant to overcome the influence of intraoperative respiratory motion and guide navigation.The present study explored the feasibility of real-time continuous image-guided ERCP.AIM To explore the feasibility of real-time continuous image-guided ERCP.METHODS We selected 23D-printed abdominal biliary tract models with different structures to simulate different patients.The ERCP environment was simulated for the biliary phantom experiment to create a navigation system,which was further tested in patients.In addition,based on the estimation of the patient’s respiratory motion,preoperative 3D biliary imaging from computed tomography of 18 patients with cholelithiasis was registered and fused in real-time with 2D fluoroscopic sequence generated by the C-arm unit during ERCP.RESULTS Continuous image-guided ERCP was applied in the biliary phantom with a registration error of 0.46 mm±0.13 mm and a tracking error of 0.64 mm±0.24mm.After estimating the respiratory motion,3D/2D registration accurately transformed preoperative 3D biliary images to each image in the X-ray image sequence in real-time in 18 patients,with an average fusion rate of 88%.CONCLUSION Continuous image-guided ERCP may be an effective approach to assist the operator and reduce the use of X-ray and contrast agents.

Natural forest ALS-TLS point cloud data registration without control points

Airborne laser scanning(ALS)and terrestrial laser scanning(TLS)has attracted attention due to their forest parameter investigation and research applications.ALS is limited to obtaining fi ne structure information below the forest canopy due to the occlusion of trees in natural forests.In contrast,TLS is unable to gather fi ne structure information about the upper canopy.To address the problem of incomplete acquisition of natural forest point cloud data by ALS and TLS on a single platform,this study proposes data registration without control points.The ALS and TLS original data were cropped according to sample plot size,and the ALS point cloud data was converted into relative coordinates with the center of the cropped data as the origin.The same feature point pairs of the ALS and TLS point cloud data were then selected to register the point cloud data.The initial registered point cloud data was fi nely and optimally registered via the iterative closest point(ICP)algorithm.The results show that the proposed method achieved highprecision registration of ALS and TLS point cloud data from two natural forest plots of Pinus yunnanensis Franch.and Picea asperata Mast.which included diff erent species and environments.An average registration accuracy of 0.06 m and 0.09 m were obtained for P.yunnanensis and P.asperata,respectively.

Automatic Extraction of the Sparse Prior Correspondences for Non-Rigid Point Cloud Registration

Non-rigid registration of point clouds is still far from stable,especially for the largely deformed one.Sparse initial correspondences are often adopted to facilitate the process.However,there are few studies on how to build them automatically.Therefore,in this paper,we propose a robust method to compute such priors automatically,where a global and local combined strategy is adopted.These priors in different degrees of deformation are obtained by the locally geometrical-consistent point matches from the globally structural-consistent region correspondences.To further utilize the matches,this paper also proposes a novel registration method based on the Coherent Point Drift framework.This method takes both the spatial proximity and local structural consistency of the priors as supervision of the registration process and thus obtains a robust alignment for clouds with significantly different deformations.Qualitative and quantitative experiments demonstrate the advantages of the proposed method.

Free form deformation and symmetry constraint‐based multimodal brain image registration using generative adversarial nets

Multi‐modal brain image registration has been widely applied to functional localisation,neurosurgery and computational anatomy.The existing registration methods based on the dense deformation fields involve too many parameters,which is not conducive to the exploration of correct spatial correspondence between the float and reference images.Meanwhile,the unidirectional registration may involve the deformation folding,which will result in the change of topology during registration.To address these issues,this work has presented an unsupervised image registration method using the free form deformation(FFD)and the symmetry constraint‐based generative adversarial networks(FSGAN).The FSGAN utilises the principle component analysis network‐based structural representations of the reference and float images as the inputs and uses the generator to learn the FFD model parameters,thereby producing two deformation fields.Meanwhile,the FSGAN uses two discriminators to decide whether the bilateral registration have been realised simultaneously.Besides,the symmetry constraint is utilised to construct the loss function,thereby avoiding the deformation folding.Experiments on BrainWeb,high grade gliomas,IXI and LPBA40 show that compared with state‐of‐the‐art methods,the FSGAN provides superior performance in terms of visual comparisons and such quantitative indexes as dice value,target registration error and computational efficiency.

Application of Opening and Closing Morphology in Deep Learning-Based Brain Image Registration

In order to improve the registration accuracy of brain magnetic resonance images(MRI),some deep learning registration methods use segmentation images for training model.How-ever,the segmentation values are constant for each label,which leads to the gradient variation con-centrating on the boundary.Thus,the dense deformation field(DDF)is gathered on the boundary and there even appears folding phenomenon.In order to fully leverage the label information,the morphological opening and closing information maps are introduced to enlarge the non-zero gradi-ent regions and improve the accuracy of DDF estimation.The opening information maps supervise the registration model to focus on smaller,narrow brain regions.The closing information maps supervise the registration model to pay more attention to the complex boundary region.Then,opening and closing morphology networks(OC_Net)are designed to automatically generate open-ing and closing information maps to realize the end-to-end training process.Finally,a new registra-tion architecture,VM_(seg+oc),is proposed by combining OC_Net and VoxelMorph.Experimental results show that the registration accuracy of VM_(seg+oc) is significantly improved on LPBA40 and OASIS1 datasets.Especially,VM_(seg+oc) can well improve registration accuracy in smaller brain regions and narrow regions.

Multimodal Medical Image Registration and Fusion for Quality Enhancement

For the last two decades,physicians and clinical experts have used a single imaging modality to identify the normal and abnormal structure of the human body.However,most of the time,medical experts are unable to accurately analyze and examine the information from a single imaging modality due to the limited information.To overcome this problem,a multimodal approach is adopted to increase the qualitative and quantitative medical information which helps the doctors to easily diagnose diseases in their early stages.In the proposed method,a Multi-resolution Rigid Registration(MRR)technique is used for multimodal image registration while Discrete Wavelet Transform(DWT)along with Principal Component Averaging(PCAv)is utilized for image fusion.The proposed MRR method provides more accurate results as compared with Single Rigid Registration(SRR),while the proposed DWT-PCAv fusion process adds-on more constructive information with less computational time.The proposed method is tested on CT and MRI brain imaging modalities of the HARVARD dataset.The fusion results of the proposed method are compared with the existing fusion techniques.The quality assessment metrics such as Mutual Information(MI),Normalize Crosscorrelation(NCC)and Feature Mutual Information(FMI)are computed for statistical comparison of the proposed method.The proposed methodology provides more accurate results,better image quality and valuable information for medical diagnoses.

A Review of Point Feature Based Medical Image Registration

Point features, as the basis of lines, surfaces, and bodies, are commonly used in medical image registration. To obtain an elegant spatial transformation of extracted feature points, many point set matching algorithms(PMs) have been developed to match two point sets by optimizing multifarious distance functions. There are ample reviews related to medical image registration and PMs which summarize their basic principles and main algorithms separately. However,to data, detailed summary of PMs used in medical image registration in different clinical environments has not been published. In this paper, we provide a comprehensive review of the existing key techniques of the PMs applied to medical image registration according to the basic principles and clinical applications. As the core technique of the PMs, geometric transformation models are elaborated in this paper, demonstrating the mechanism of point set registration. We also focus on the clinical applications of the PMs and propose a practical classification method according to their applications in different clinical surgeries. The aim of this paper is to provide a summary of pointfeaturebased methods used in medical image registration and to guide doctors or researchers interested in this field to choose appropriate techniques in their research.

SuperFusion: A Versatile Image Registration and Fusion Network with Semantic Awareness

Image fusion aims to integrate complementary information in source images to synthesize a fused image comprehensively characterizing the imaging scene. However, existing image fusion algorithms are only applicable to strictly aligned source images and cause severe artifacts in the fusion results when input images have slight shifts or deformations. In addition,the fusion results typically only have good visual effect, but neglect the semantic requirements of high-level vision tasks.This study incorporates image registration, image fusion, and semantic requirements of high-level vision tasks into a single framework and proposes a novel image registration and fusion method, named Super Fusion. Specifically, we design a registration network to estimate bidirectional deformation fields to rectify geometric distortions of input images under the supervision of both photometric and end-point constraints. The registration and fusion are combined in a symmetric scheme, in which while mutual promotion can be achieved by optimizing the naive fusion loss, it is further enhanced by the mono-modal consistent constraint on symmetric fusion outputs. In addition, the image fusion network is equipped with the global spatial attention mechanism to achieve adaptive feature integration. Moreover, the semantic constraint based on the pre-trained segmentation model and Lovasz-Softmax loss is deployed to guide the fusion network to focus more on the semantic requirements of high-level vision tasks. Extensive experiments on image registration, image fusion,and semantic segmentation tasks demonstrate the superiority of our Super Fusion compared to the state-of-the-art alternatives.The source code and pre-trained model are publicly available at https://github.com/Linfeng-Tang/Super Fusion.

A Correntropy-based Affine Iterative Closest Point Algorithm for Robust Point Set Registration

The iterative closest point(ICP)algorithm has the advantages of high accuracy and fast speed for point set registration,but it performs poorly when the point set has a large number of noisy outliers.To solve this problem,we propose a new affine registration algorithm based on correntropy which works well in the affine registration of point sets with outliers.Firstly,we substitute the traditional measure of least squares with a maximum correntropy criterion to build a new registration model,which can avoid the influence of outliers.To maximize the objective function,we then propose a robust affine ICP algorithm.At each iteration of this new algorithm,we set up the index mapping of two point sets according to the known transformation,and then compute the closed-form solution of the new transformation according to the known index mapping.Similar to the traditional ICP algorithm,our algorithm converges to a local maximum monotonously for any given initial value.Finally,the robustness and high efficiency of affine ICP algorithm based on correntropy are demonstrated by 2D and 3D point set registration experiments.

Global pattern and trends of colorectal cancer survival: a systematic review of population-based registration data

This review will describe the global patterns and trends of colorectal cancer survival,using data from the population-based studies or cancer registration.We performed a systematic search of China National Knowledge Infrastructure(CNKI),Wanfang Data,PubMed,Web of Science,EMBASE,and SEER and collected all population-based survival studies of colorectal cancer(up to June 2020).Estimates of observed and relative survival rates of colorectal cancer by sex,period,and country were extracted from original studies to describe the temporal patterns and trends from the late 1990s to the early 21st century.Globally,5-year observed survival rates were higher in Seoul,Republic of Korea(1993–1997;56.8%and 54.3%for colon and rectum cancers,respectively),Zhejiang province(2005–2010;52.9%for colon cancer),Tianjin(1991–1999;52.5%for colon cancer),Shanghai(2002–2006;50.0%for rectum cancer)of China,and in Japan(1993–1996,59.6%for colorectal cancer).Five-year relative survival rates of colorectal cancer in the Republic of Korea(2010–2014),Queensland,Australia(2005–2012),and the USA(2005–2009)ranked at relatively higher positions compared to other countries.In general,colorectal cancer survival rates are improving over time worldwide.Sex disparities in survival rates were also observed in the colon,rectum,and colorectal cancers in most countries or regions.The poorest age-specific 5-year relative survival rate was observed in patients>75 years of age.In conclusion,over the past 3 decades,colorectal cancer survival has gradually improved.Geographic variations,sex differences,and age gradients were also observed globally in colorectal cancer survival.Further studies are therefore warranted to investigate the prognostic factors of colorectal cancer.

Total Variation Constrained Non-Negative Matrix Factorization for Medical Image Registration

This paper presents a novel medical image registration algorithm named total variation constrained graphregularization for non-negative matrix factorization(TV-GNMF).The method utilizes non-negative matrix factorization by total variation constraint and graph regularization.The main contributions of our work are the following.First,total variation is incorporated into NMF to control the diffusion speed.The purpose is to denoise in smooth regions and preserve features or details of the data in edge regions by using a diffusion coefficient based on gradient information.Second,we add graph regularization into NMF to reveal intrinsic geometry and structure information of features to enhance the discrimination power.Third,the multiplicative update rules and proof of convergence of the TV-GNMF algorithm are given.Experiments conducted on datasets show that the proposed TV-GNMF method outperforms other state-of-the-art algorithms.

Incoherent digital holographic spectral imaging with high accuracy of image pixel registration

Fresnel incoherent correlation holography(FINCH) is a unique three-dimensional(3D) imaging technique which has the advantages of scanning-free,high resolution,and easy matching with existing mature optical systems.In this article,an incoherent digital holographic spectral imaging method with high accuracy of spectral reconstruction based on liquid crystal tunable filter(LCTF) and FINCH is proposed.Using the programmable characteristics of spatial light modulator(SLM),a series of phase masks,none of whose focal lengths changes with wavelength,is designed and made.For each wavelength of LCTF output,SLM calls three phase masks with different phase constants at the corresponding wavelength,and CCD records three holograms.The spectral images obtained by this method have a constant magnification,which can achieve pixel-level image registration,restrain image registration errors,and improve spectral reconstruction accuracy.The results show that this method can not only obtain the 3D spatial information and spectral information of the object simultaneously,but also have high accuracy of spectral reconstruction and excellent color reproducibility.

Registration Based on ORB and FREAK Features for Augmented Reality Systems

This paper proposes a novel registration method for augmented reality(AR) systems based on Oriented FAST and RotatedBRIEF(ORB) and Fast Retina Keypoint(FREAK) natural features. In the proposed ORBFREAK method, feature extraction is implemented based on the combination of ORB and FREAK, and the feature points are matched using Hamming distance. To get good matching points, cross-checks and least median squares are used to perform outlier filtration, and camera pose is estimated using the matched points. Finally, AR is rendered.Experiments show that the proposed method improves the speed of registration to be in real time; the proposed method can accurately register the target object under the circumstances of partial occlusion of the object; and it also can overcome the effects of rotation, scale change, ambient light and distance.

Using shape contexts method for registration of contra lateral breasts in thermal images

AIM: To achieve symmetric boundaries for left and right breasts boundaries in thermal images by registration. METHODS: The proposed method for registration consists of two steps. In the first step, shape context, an approach as presented by Belongie and Malik was applied for registration of two breast boundaries. The shape context is an approach to measure shape similarity. Two sets of finite sample points from shape contours of two breasts are then presented. Consequently, the correspondences between the two shapes are found. By finding correspondences, the sample point which has the most similar shape context is obtained. RESULTS: In this study, a line up transformation which maps one shape onto the other has been estimated in order to complete shape. The used of a thin plate spline permitted good estimation of a plane transformation which has capability to map unselective points from one shape onto the other. The obtained aligningtransformation of boundaries points has been applied successfully to map the two breasts interior points. Some of advantages for using shape context method in this work are as follows:(1) no special land marks or key points are needed;(2) it is tolerant to all common shape deformation; and(3) although it is uncomplicated and straightforward to use, it gives remarkably powerful descriptor for point sets significantly upgrading point set registration. Results are very promising. The proposed algorithm was implemented for 32 cases. Boundary registration is done perfectly for 28 cases.CONCLUSION: We used shape contexts method that is simple and easy to implement to achieve symmetric boundaries for left and right breasts boundaries in thermal images.

RANSACs for 3D Rigid Registration:A Comparative Evaluation

Estimating an accurate six-degree-of-freedom(6-Do F)pose from correspondences with outliers remains a critical issue to 3D rigid registration.Random sample consensus(RANSAC)and its variants are popular solutions to this problem.Although there have been a number of RANSAC-fashion estimators,two issues remain unsolved.First,it is unclear which estimator is more appropriate to a particular application.Second,the impacts of different sampling strategies,hypothesis generation methods,hypothesis evaluation metrics,and stop criteria on the overall estimators remain ambiguous.This work fills these gaps by first considering six existing RANSAC-fashion methods and then proposing eight variants for a comprehensive evaluation.The objective is to thoroughly compare estimators in the RANSAC family,and evaluate the effects of each key stage on the eventual 6-Do F pose estimation performance.Experiments have been carried out on four standard datasets with different application scenarios,data modalities,and nuisances.They provide us with input correspondence sets with a variety of inlier ratios,spatial distributions,and scales.Based on the experimental results,we summarize remarkable outcomes and valuable findings,so as to give practical instructions to real-world applications,and highlight current bottlenecks and potential solutions in this research realm.

Coordinate registration algorithms for over-the-horizon radar

Different from the other conventional radars, the over the horizon radar （OTHR） faces complicated nonlinear coordinate transform due to electromagnetic wave propagation and reflection in ionospheres. A significant problem is the phenomenon of multi-path propagation. Considering it, the coordinate registration algorithms of planar measurement model and spherical measurement model are respectively derived in detail. Noticeably, a new transforming expression of apparent azimuth and an integrated form of transforming expressions from measurement vector to ground state vector in coordinate registration algorithm of spherical measurement model are proposed. And then simulations are made to verify the correctness of the proposed algorithms and expression. Besides this, the transforming error rate of slant range, Doppler and apparent azimuth of the two kinds of models are given respectively. Then the quantitative analysis of error rate is also given. It can be drawn a conclusion that the coordinate registration algorithms of planar measurement model and spherical measurement model are both correct.

HIGH-ACCURACY BAND TO BAND REGISTRATION METHOD FOR MULTI-SPECTRAL IMAGES OF HJ-1A/B

Band-to-band registration accuracy is an important parameter of multispectral data. A novel band-to-band registration approach with high precision is proposed for the multi-spectral images of HJ-1A/B. Firstly, the main causes resulted in misregistration are analyzed, and a high-order polynomial model is proposed. Secondly, a phase fringe filtering technique is employed to Phase Correlation Method based on Singular Value Decomposition (SVD-PCM) for reducing the noise in phase difference matrix. Then, experiments are carried out to build nonlinear registration models, and images of green band and red band are aligned to blue band with an accuracy of 0.1 pixels, while near infrared band with an accuracy of 0.2 pixels.

THE STUDY ON AUTOMATIC REGISTRATION OF INSAR IMAGES

Interferometric Synthetic Aperture Radar (InSAR) allows production of high resolution DEM and detection of small earth motions using multiple pass SAR data sets obtained by remote sensing satellite. But the whole process has not yet reached sufficient robustness to warrant automated DEM production as commonly produced by stereovision with optical images. The automatic algorithm for precision registration is one of the bottlenecks in InSAR data processing. In this paper, an automatic approach with multi-step image matching algorithm is presented. All procedures are automatically implemented. The experiment is carried out successfully with SIR-C/L-band InSAR data. The triangular piecewise rectification is also advanced in reducing local distortion between the images and processing the large scene image. The primary result has prospect in the precision registration for the repeat-track InSAR data and reveals the potential of the presented automatic strategy.

Optical and SAR image registration based on improved nonsubsampled wavelet transform for sea islands

Homologous feature point extraction is a key problem in the optical and synthetic aperture radar （SAR） image registration for islands. A new feature point extraction method using a threshold shrink operator and non-subsampled wavelet transform （TSO-NSWT） for optical and SAR image registration was proposed. Moreover, the matching for this proposed feature was different from the traditional feature matching strategies and was performed using a similarity measure computed from neighborhood circles in low-frequency bands. Then, a number of reliably matched couples with even distributions were obtained, which assured the accuracy of the registration. Application of the proposed algorithm to SPOT-5 （multi-spectral） and YG-1 （SAR） images showed that a large number of accurately matched couples could be identified. Additionally, both of the root mean square error （RMSE） patterns of the registration parameters computed based on the TSO-NSWT algorithm and traditional NSWT algorithm were analyzed and compared, which further demonstrated the effectiveness of the proposed algorithm. The algorithm can supply the crucial step for island image registration and island recognition.