A Review of Point Feature Based Medical Image Registration

Point features, as the basis of lines, surfaces, and bodies, are commonly used in medical image registration. To obtain an elegant spatial transformation of extracted feature points, many point set matching algorithms(PMs) have been developed to match two point sets by optimizing multifarious distance functions. There are ample reviews related to medical image registration and PMs which summarize their basic principles and main algorithms separately. However,to data, detailed summary of PMs used in medical image registration in different clinical environments has not been published. In this paper, we provide a comprehensive review of the existing key techniques of the PMs applied to medical image registration according to the basic principles and clinical applications. As the core technique of the PMs, geometric transformation models are elaborated in this paper, demonstrating the mechanism of point set registration. We also focus on the clinical applications of the PMs and propose a practical classification method according to their applications in different clinical surgeries. The aim of this paper is to provide a summary of pointfeaturebased methods used in medical image registration and to guide doctors or researchers interested in this field to choose appropriate techniques in their research.

Fast Mutual Information Registration Method of 3-D Medical Image

Currently the voxel based registration methods have been used widely such as the well known mutual information (MI). Although the accuracy of their results is plausible, the registration procedure is slow. This paper proposed some methods to rigid registration based on mutual information, aiming for an acceleration of the registration process without significantly loss of precision in the final results. The efficiency of these methods is examined by registration of CT MR and PET MR. Experimental results show that the speedup is effective and efficient. By using the fast methods, the registration of 3 D medical image could also be implemented on PC rapidly.

Application of Opening and Closing Morphology in Deep Learning-Based Brain Image Registration

In order to improve the registration accuracy of brain magnetic resonance images(MRI),some deep learning registration methods use segmentation images for training model.How-ever,the segmentation values are constant for each label,which leads to the gradient variation con-centrating on the boundary.Thus,the dense deformation field(DDF)is gathered on the boundary and there even appears folding phenomenon.In order to fully leverage the label information,the morphological opening and closing information maps are introduced to enlarge the non-zero gradi-ent regions and improve the accuracy of DDF estimation.The opening information maps supervise the registration model to focus on smaller,narrow brain regions.The closing information maps supervise the registration model to pay more attention to the complex boundary region.Then,opening and closing morphology networks(OC_Net)are designed to automatically generate open-ing and closing information maps to realize the end-to-end training process.Finally,a new registra-tion architecture,VM_(seg+oc),is proposed by combining OC_Net and VoxelMorph.Experimental results show that the registration accuracy of VM_(seg+oc) is significantly improved on LPBA40 and OASIS1 datasets.Especially,VM_(seg+oc) can well improve registration accuracy in smaller brain regions and narrow regions.

Point Reg Net: Invariant Features for Point Cloud Registration Using in Image-Guided Radiation Therapy

In image-guided radiation therapy, extracting features from medical point cloud is the key technique for multimodality registration. This novel framework, denoted Control Point Net (CPN), provides an alternative to the common applications of manually designed keypoint descriptors for coarse point cloud registration. The CPN directly consumes a point cloud, divides it into equally spaced 3D voxels and transforms the points within each voxel into a unified feature representation through voxel feature encoding (VFE) layer. Then all volumetric representations are aggregated by Weighted Extraction Layer which selectively extracts features and synthesize into global descriptors and coordinates of control points. Utilizing global descriptors instead of local features allows the available geometrical data to be better exploited to improve the robustness and precision. Specifically, CPN unifies feature extraction and clustering into a single network, omitting time-consuming feature matching procedure. The algorithm is tested on point cloud datasets generated from CT images. Experiments and comparisons with the state-of-the-art descriptors demonstrate that CPN is highly discriminative, efficient, and robust to noise and density changes.

Robust Image Registration Based on Mutual Information Measure

A new implementation of the image registration algorithm based on the mutual information is presented for the case of medical images. The registration is achieved if the maximum of the mutual information is attained. In this maximization process optimal values of five parameters of an affine transformation are searched.

MUTUAL INFORMATION BASED 3D NON-RIGID REGISTRATION OF CT/MR ABDOMEN IMAGES

A mutual information based 3D non-rigid registration approach was proposed for the registration of deformable CT/MR body abdomen images. The Parzen Windows Density Estimation (PWDE) method is adopted to calculate the mutual information between the two modals of CT and MRI abdomen images. By maximizing MI between the CT and MR volume images, the overlapping part of them reaches the biggest, which means that the two body images of CT and MR matches best to each other. Visible Human Project (VHP) Male abdomen CT and MRI Data are used as experimental data sets. The experimental results indicate that this approach of non-rigid 3D registration of CT/MR body abdominal images can be achieved effectively and automatically, without any prior processing procedures such as segmentation and feature extraction, but has a main drawback of very long computation time.

Study of three-dimensional PET and MR image registration based on higher-order mutual information

Mutual information has currently been one of the most intensivelyresearched measures. It has been proven to be accurate and effective registrationmeasure. Despite the general promising results, mutual information sometimes mightlead to misregistration because of neglecting spatial information and treating intensityvariations with undue sensitivity. In this paper, an extension of mutual informationframework was proposed in which higher-order spatial information regarding imagestructures was incorporated into the registration processing of PET and MR. Thesecond-order estimate of mutual information algorithm was applied to the registrationof seven patients. Evaluation from Vanderbilt University and our visual inspectionshowed that sub-voxel accuracy and robust results were achieved in all cases withsecond-order mutual information as the similarity measure and with Powell's multi-dimensional direction set method as optimization strategy.

Nonlinear Registration of Brain Magnetic Resonance Images with Cross Constraints of Intensity and Structure

Many deep learning-based registration methods rely on a single-stream encoder-decoder network for computing deformation fields between 3D volumes.However,these methods often lack constraint information and overlook semantic consistency,limiting their performance.To address these issues,we present a novel approach for medical image registration called theDual-VoxelMorph,featuring a dual-channel cross-constraint network.This innovative network utilizes both intensity and segmentation images,which share identical semantic information and feature representations.Two encoder-decoder structures calculate deformation fields for intensity and segmentation images,as generated by the dual-channel cross-constraint network.This design facilitates bidirectional communication between grayscale and segmentation information,enabling the model to better learn the corresponding grayscale and segmentation details of the same anatomical structures.To ensure semantic and directional consistency,we introduce constraints and apply the cosine similarity function to enhance semantic consistency.Evaluation on four public datasets demonstrates superior performance compared to the baselinemethod,achieving Dice scores of 79.9%,64.5%,69.9%,and 63.5%for OASIS-1,OASIS-3,LPBA40,and ADNI,respectively.

SeRN:A Two-Stage Framework of Registration for Semi-Supervised Learning for Medical Images

Significant breakthroughs in medical image registration have been achieved using deep neural networks(DNNs).However,DNN-based end-to-end registration methods often require large quantities of data or adequate annotations for training.To leverage the intensity information of abundant unlabeled images,unsupervised registration methods commonly employ intensity-based similarity measures to optimize the network parameters.However,finding a sufficiently robust measure can be challenging for specific registration applications.Weakly supervised registration methods use anatomical labels to estimate the deformation between images.High-level structural information in label images is more reliable and practical for estimating the voxel correspondence of anatomic regions of interest between images,whereas label images are extremely difficult to collect.In this paper,we propose a two-stage semi-supervised learning framework for medical image registration,which consists of unsupervised and weakly supervised registration networks.The proposed semi-supervised learning framework is trained with intensity information from available images,label information from a relatively small number of labeled images and pseudo-label information from unlabeled images.Experimental results on two datasets(cardiac and abdominal images)demonstrate the efficacy and efficiency of this method in intra-and inter-modality medical image registrations,as well as its superior performance when a vast amount of unlabeled data and a small set of annotations are available.Our code is publicly available at at https://github.com/jdq818/SeRN.

Transformers in medical image analysis

Transformers have dominated the field of natural language processing and have recently made an impact in the area of computer vision.In the field of medical image analysis,transformers have also been successfully used in to full-stack clinical applications,including image synthesis/reconstruction,registration,segmentation,detection,and diagnosis.This paper aimed to promote awareness of the applications of transformers in medical image analysis.Specifically,we first provided an overview of the core concepts of the attention mechanism built into transformers and other basic components.Second,we reviewed various transformer architectures tailored for medical image applications and discuss their limitations.Within this review,we investigated key challenges including the use of transformers in different learning paradigms,improving model efficiency,and coupling with other techniques.We hope this review would provide a comprehensive picture of transformers to readers with an interest in medical image analysis.

Rapid and robust medical image elastic registration using mean shift algorithm

In landmark-based image registration, estimating the landmark correspondence plays an important role. In this letter, a novel landmark correspondence estimation technique using mean shift algorithm is proposed. Image corner points are detected as landmarks and mean shift iterations are adopted to find the most probable corresponding point positions in two images. Mutual information between intensity of two local regions is computed to eliminate mis-matching points to improve the stability of corresponding estimation correspondence landmarks is exact. The proposed experiments of various mono-modal medical images. Multi-level estimation （MLE） technique is proposed Experiments show that the precision in location of technique is shown to be feasible and rapid in the

基于轮廓点相似性测度的2D-3D医学图像配准算法

针对传统配准算法无法适用于成像模糊、对比度低的X光医学图像的问题,本文提出一种基于轮廓点相似性测度的配准技术.首先引入分块双阈值增强策略来提取DRR图像和X光图像的边缘轮廓信息;其次,采用高斯加权欧氏距离计算图像轮廓的相似度;最后通过平衡优化器算法进行迭代优化,得到最优的位姿参数.实验结果表明:本文算法能够精确提取模糊X光图像的边缘轮廓信息,而且可以准确评估其与CT数据的相似度,平均配准成功率超过94%,算法效率和鲁棒性优于传统算法,可用于医疗诊断、放射疗法、图像引导手术等医学活动.

Explainable Conformer Network for Detection of COVID-19 Pneumonia from Chest CT Scan: From Concepts toward Clinical Explainability

The early implementation of treatment therapies necessitates the swift and precise identification of COVID-19 pneumonia by the analysis of chest CT scans.This study aims to investigate the indispensable need for precise and interpretable diagnostic tools for improving clinical decision-making for COVID-19 diagnosis.This paper proposes a novel deep learning approach,called Conformer Network,for explainable discrimination of viral pneumonia depending on the lung Region of Infections(ROI)within a single modality radiographic CT scan.Firstly,an efficient U-shaped transformer network is integrated for lung image segmentation.Then,a robust transfer learning technique is introduced to design a robust feature extractor based on pre-trained lightweight Big Transfer(BiT-L)and finetuned on medical data to effectively learn the patterns of infection in the input image.Secondly,this work presents a visual explanation method to guarantee clinical explainability for decisions made by Conformer Network.Experimental evaluation of real-world CT data demonstrated that the diagnostic accuracy of ourmodel outperforms cutting-edge studies with statistical significance.The Conformer Network achieves 97.40% of detection accuracy under cross-validation settings.Our model not only achieves high sensitivity and specificity but also affords visualizations of salient features contributing to each classification decision,enhancing the overall transparency and trustworthiness of our model.The findings provide obvious implications for the ability of our model to empower clinical staff by generating transparent intuitions about the features driving diagnostic decisions.

三维图像配准技术构建膝关节及周围软组织曲面数字化模型

背景:随着现代工业、现代国防以及现代医疗康复产业的发展,仿生设计与制造技术蓬勃兴起,该技术常以各类生物体及其组织为研究对象,模仿其几何外形、力学性能等指标,进而得以逆向设计与制造各类人工合成产品,其中几何外形特别是曲面建模技术的有效仿生是其中的关键一环。目的:以人体膝关节为例探讨曲面建模实践应用技术,建立人体膝关节几何解剖数字化模型,为膝关节生物力学特性的研究提供支持,同时为仿生设计的复杂曲面建模提供了一种有效的教学与研究模式。方法:利用膝关节CT扫描图像和MRI图像,通过医学影像处理技术分别获取骨组织及软组织轮廓的点云数据及其逼近曲线,并分别重建骨组织及软组织的膝关节实体模型。最后基于三维图像配准技术,将各软组织组装配于骨组织实体模型上,形成整体膝关节三维模型。结果与结论:通过3D医学影像技术构建了同时包含各骨组织和韧带、半月板、肌腱等主要软组织在内的膝关节复杂几何曲面解剖数字化模型。

基于注意力机制无监督心脏超声序列图像配准

针对基于传统非刚性医学图像配准的心脏超声序列图像配准方法缺乏自动性及配准速度慢、准确率较低的问题,将基于深度学习的医学图像配准算法应用于心脏超声序列图像配准,通过引入通道注意力机制,构建由注意力机制模块、Unet卷积神经网络模块及空间转换模块STN构成的配准模型.实验选取不同的相似性损失函数和平滑损失函数,对比VoxelMorph配准模型,相关配准性能指标都有不同程度的改进,DICE指标提升0.42%,MI指标提升2.5%,SSIM提升3.7%,NRMSE减小9%,表明配准模型的有效性.从配准效果及配准时间分析,配准模型基本可以满足心脏超声序列图像配准的实时性需求,具有一定的临床应用价值.

视觉Transformer在医学图像分析中的应用研究综述

深度自注意力网络(Transformer)对输入信息全局特征和长距离相关性具有天然良好的建模能力,其与卷积神经网络(CNN)的归纳偏置特性具有较强互补性。受其在自然语言处理领域取得巨大成功的启发,Transformer已被广泛引入到计算机视觉各项任务特别是医学图像分析领域并已取得了不俗表现。对Transformer与自然图像结合的典型工作进行介绍,根据视觉Transformer在医学图像分割、医学图像分类以及医学图像配准等子领域对相关工作按照不同病灶及部位进行了整理和归纳,重点对一些代表性研究工作的实现思想进行了详细分析。对现有研究工作进行了讨论并对未来方向进行了展望,以期为该领域的进一步深入研究提供参考。

基于深度学习的直肠癌图像配准技术研究进展

图像配准在直肠癌诊断、手术引导及放射性治疗等诊疗场景中具有重要的应用价值。传统的医学图像配准方法非常耗时,不能满足临床实时性的需求。基于深度学习的配准技术,以其计算速率快、适用范围广引起广泛关注。文章首先从两类深度学习的图像配准方法进行分析,并对直肠癌图像配准进行阐述;然后,对深度学习图像配准以及直肠癌医学配准的4个挑战进行讨论;最后,展望未来的研究方向。

LK-CAUNet:基于交叉注意的大内核多尺度可变形医学图像配准网络

经典的UNet网络可用于预测全分辨率空间域的密集位移场,在医学图像配准中取得了巨大成功。但对大变形的三维图像配准,还存在运行时间长、无法有效保持拓扑结构、空间特征易丢失等缺点。为此,提出一种基于交叉注意的大内核多尺度可变形医学图像配准网络(large kernel multi-scale deformable medical image registration network based on cross-attention,LK-CAUNet)。在经典UNet模型基础上,通过引入交叉注意力模块,实现高效、多层次的语义特征融合;配备大内核非对称并行卷积,使其具有多尺度特征和对复杂结构的学习能力;通过加入平方和缩放模块,实现拓扑守恒和变换可逆。基于脑部MRI数据集,将LK-CAUNet与18种经典图像配准模型进行了比较,结果表明,LK-CAUNet的配准性能较其他模型有明显提升,其Dice得分较TransMorph配准方法提高了8%,而参数量仅为TransMorph的1/5。

形变医学图像配准方法设计与仿真

利用目前方法对形变医学图像进行配准时,没有提取形变医学图像特征,存在特征点获取结果与实际结果相差大、医学图像配准效果差和医学图像配准时间长的问题。为此提出基于角点检测与SIFT的形变医学图像配准方法。采用角点检测与SIFT相结合的方法对医学图像的特征点进行提取,在图像特征提取前,优先对尺度空间的极值点进行检测,其次生成角点特征,通过检测结果与最终特征点的方向完成医学图像特征点的提取,提升了医学图像配准精度。将提取的特征输入到构建的深度学习模型中,根据提取特征的训练及损失函数的优化实现形变医学图像配准。实验结果表明,通过对上述方法进行特征点获取结果与实际结果对比测试、医学图像配准效果测试和配准时间测试,验证了上述方法的准确性与有效性。