应用图像块和全卷积神经网络的肩关节MRI自动分割

Automatic segmentation of shoulder joint in MRI by using patch-wise and full-image fully convolutional networks

目的 MRI正逐步代替CT进行骨头与关节的检查,肩关节MRI中骨结构的精确自动分割对于骨损伤和疾病的度量与诊断至关重要,现有骨头分割算法无法做到不用任何先验知识进行自动分割,且通用性和精准度相对较低,为此提出一种基于图像块和全卷积神经网络(PCNN和FCN)相结合的自动分割算法。方法首先建立4个分割模型,包括3个基于U-Net的骨头分割模型(肱骨分割模型、关节骨分割模型、肱骨头和关节骨作为整体的分割模型)和一个基于块的Alex Net分割模型;然后使用4个分割模型来获取候选的骨头区域,并通过投票的方式准确检测到肱骨和关节骨的位置区域;最后在检测到的骨头区域内进一步使用Alex Net分割模型,从而分割出精确度在像素级别的骨头边缘。结果实验数据来自美国哈佛医学院/麻省总医院骨科的8组病人,每组扫描序列包括100片左右图像,都已经分割标注。5组病人用于训练和进行五倍的交叉验证,3组病人用于测试实际的分割效果,其中Dice Coefficient、Positive Predicted Value(PPV)和Sensitivity平均准确率分别达到0. 92±0. 02、0. 96±0. 03和0. 94±0. 02。结论本文方法针对小样本的病人数据集,仅通过2维医学图像上的深度学习,可以得到非常精确的肩关节分割结果。所提算法已经集成到我们开发的医学图像度量分析平台"3DQI",通过该平台可以展示肩关节骨头3D分割效果,给骨科医生提供临床的诊断指导作用。同时,所提算法框架具有一定的通用性,适应于小样本数据下CT和MRI中特定器官和组织的精确分割。

Objective MRI uses the principle of nuclear magnetic resonance. In addition, MRI is safe to use and has a high soft tissue resolution. CT is being gradually replaced by MRI for checking bones and joints. The automated detection and segmentation of shoulder joint structures in MRI are extremely important for measuring and diagnosing bone injuries and diseases. In MRI, the internal bone region and the air, fat, and some soft tissue in it show similar gray and black features and a low image signal to noise ratio and partial volume effect. Thus, the automatic and accurate segmentation of the clinically valuable glenoid and humeral head in the shoulder joint is difficult. Common conventional bone segmentation algorithms, such as region growing or level set, cannot be implemented without prior knowledge, and their generality and accuracy are relatively low. Although various deep learning algorithms have been applied to the segmentation of medical images, such as MRI and CT, a successful segmentation is almost impossible with only one deep network without post-processing. To the best of our knowledge, few studies on the use of deep learning to segment bones in MRI have been conducted, and no study has been conducted about shoulder segmentation. Two deep learning networks, namely, patch-based and fully convolutional networks （ PCNN and FCN） , are employed for the automated detection and segmentation of the shoulder joint structure in MRI. Method First, four segmentation models are built, including three U-Net based models （ glenoid segmentation model, humeral head segmentation model, glenoid and humeral head segmentation model） and one patch-based AlexNet segmentation model. The network depth of U-Net is 3, and the number of features in the first layer is 16. Edge expansion is performed by mirroring to ensure that the resolution of the output segmentation map is consistent with the original image because the resolution of the input image is reduced after passing through U-Net. The traditional AlexNet has a three-channel RGB input, but MRI is a grayscale image, and thus the RGB channel values are the same. Three channels can be added to an image by using the three window-level mappings or three different resolutions. However, no performance improvement is observed relative to the channel test, and thus, we adjust the input to one channel. Then, the four segmentation models are used to obtain the candidate bone regions from which the correct glenoid and humeral head locations and regions are obtained by voting. However, false bone regions still exist. Given that the signal intensity of the bone is near the fat and some soft tissues, which are easily misjudged as glenoid and humeral head in partial shapes and positions, and due to the complex shape and wider variation range of the humeral head, the diversity of morphologies easily leads to the misinterpretation of noise or fat as a humeral head. These false bone regions are filtered by location information, and the missing bone objects are calculated by inter-frame prediction due to the continuity of MRI scanning in the time direction. Finally, the AlexNet model is used to segment the edge of the bone with accuracy at the pixel level. Result The experimental data are derived from eight groups of patients of Harvard Medical School/Massachusetts General Hospital of the United States. Each scan sequence includes approximately 100 images with marked bone edge labels. Five groups of patients are used for training and five-fold cross-validation, and three groups are used to test the actual segmentation results. The Dice coefficient, positive predicted value, and sensitivity average accuracy are 0. 92 ± 0.02, 0. 96 ± 0. 03, and 0. 94 ± 0. 02, respectively. Experimental results show that the segmentation accuracy is very high, which is basically consistent with the results of artificial segmentation. The segmentation accuracy also exceeds the average artificial annotation in a considerable part of the images from observation. In practical segmentation applications, training and segmentation are generally performed at the service end of the GPU device, and the segmentation result is displayed at the client end. For medical institutions, the operation is usually performed in a local area network. For a slice of a patient ＇ s MRI sequence, the over- all time from the segmentation request of the client to the segmentation result from the server is approximately 1.2 seconds, meeting the real-time requirements of the application. In many cases, the scanned images of a group of patients are uniformly processed offline on the server side, and the segmentation results are saved. The segmentation results can be retrieved when the client loads the patient data. In this case, no real-time performance requirement exists, which is common in ap- plication modes. From the experimental dataset, our sample set is very limited, that is, 8 sets of patient use cases. This finding indicates that a highly effective predictive effect can be achieved if the methods for obtaining image blocks and data argument are appropriate. Conclusion The model ensemble method obtained by voting is used to accurately locate the glenoid and humeral head bone in the shoulder joint. Four types of segmentation modeling are performed. The spatial consistency of the image sequence is used to predict the incorrectly deleted area. Then, PCNN segmentation is employed by the local perception and features in the located bone region of interest. Although the patients＇ datasets are quite small, accurate shoulder joint segmentation results are obtained. The proposed algorithm has been integrated into 3DQI, a medical image measurement and analysis platform we developed, which can demonstrate 3D segmentation of shoulder bones and provide clinical diagnosis and guidance to orthopedists. With the deepening cooperation with hospitals and the increasing number of MRI samples, we can test and analyze the three-dimensional segmentation based on deep learning and compare the segmentation results with the two-dimensional operation in future studies.