融合空间相关性特征的乳腺组织病理全切片分类

Classification of whole slide images of breast histopathology based on spatial correlation characteristics

目的病理学检查是明确乳腺癌诊断及肿瘤类型的金标准。深度神经网络广泛应用于乳腺病理全切片的诊断工作并取得了明显进展,但是现有大多数工作只是将全切片切割成小图像块,对每个图像块进行单独处理,没有考虑它们之间的空间信息。为此,提出了一种融合空间相关性特征的乳腺组织病理全切片分类方法。方法首先基于卷积神经网络对病理图像块进行预测,并提取每个图像块有代表性的深层特征,然后利用特征融合将图像块及其周围图像的特征进行聚合,以形成具有空间相关性的块描述符,最后将全切片图像中最可疑的块描述符传递给循环神经网络,以预测最终的全切片级别的分类。结果本文构建了一个经过详细标注的乳腺病理全切片数据集,并在此数据集上进行良性/恶性二分类实验。在自建的数据集中与3种全切片分类方法进行了比较。结果表明,本文方法的分类精度达到96.3%,比未考虑空间相关性的方法高出了1.9%,与基于热力图特征和基于空间性和随机森林的方法相比,分类精度分别高出8.8%和1.3%。结论本文提出的乳腺组织病理全切片识别方法将空间相关性特征融合和RNN分类集成到一个统一模型,有助于提高图像识别准确率,为病理图像诊断工作提供了高效的辅助诊断工具。

Objective Pathological examination can be as the“gold standard”to interpret breast cancer diagnosis and its tumor types.To improve the treatment effect of breast cancer,accurate interpretation is beneficial to clarify the type of the disease early and effectively.However,it is time-consuming to check the pathological whole slide images(WSIs)manu⁃ally,and the diagnosis result is easily affected by personal experience.Due to computer vision-based convolutional neural networks(CNNs)are applied to the classification task of histopathological images computer-aided diagnostic(CAD)tech⁃niques for digital images of histopathology has been developing intensively.However,it is still challenged to split largesized WSIs into small patches and each patch is processed individually without the spatial information between them.Also,the information-involved of the patches learned is not utilized in the small size patch feature aggregation process.To resolve these problems,we develop a spatial correlation-based classification method for breast histopathology images,which can re-identify the WSIs classification problem through deep learning feature fusion and recurrent neural network(RNN)based classification.Method Our demonstration consists of four aspects:1)pre-processing operation of WSIs,2)CNN-based breast patch prediction,3)image patch feature fusion,and 4)RNN-based WSIs classification.First,it is focused on a preprocessing operation on whole slide images of breast pathology,using a sliding window to cut the WSIs into patches of a suitable size in terms of a CNN.Second,CNN-based patch prediction is used to predict the possibility of cancer in each patch and each patch is encoded as a fixed-length feature.To reduce unpredictable class of cancerous patch being classi⁃fied as non-cancerous patch,the ResNet34 is used as the patch classification model and penalty factors is added in Focal loss.Third,the feature fusion of image blocks can be recognized as a feature fusion niche that takes the entire grid of patches as input and the spatial correlation of patches and their surrounding patches are as feature fusion of all patches within the grid,thus block descriptors are formed.Feature fusion methods are involved in,including 1)Weight,2)Max,3)Avg,4)Norm3,and 5)WeightNorm3.Finally,the RNN-based WSIs classification is used to pass the block feature descriptors with high probability of cancer in each WSIs to the RNN,and the RNN model is used to learn the sequence rela⁃tionship between the image block feature sequences.It can expand the processing field of view of the classification model to multiple Blocks and optimize the classification accuracy of breast pathology WSIs.Additionally,we design and develop an online recognition system for assistance.Result A detailed annotated whole slide images dataset of breast pathology is con⁃structed,and benign/malignant dichotomous classification experiments are carried out on this dataset.The results are com⁃pared to three WSIs classification methods in the self-constructed dataset,and it shows that the classification accuracy of the method can be reached to 96.3%,which is 1.9%higher than non-spatial correlation method.The classification accu⁃racy of each is 8.8%and 1.3%higher compared to heat map features-based methods and spatial correlation and random forest-related methods.Conclusion The proposed breast pathology recognition method can improve image recognition accu⁃racy and provide an efficient diagnostic aid for pathology image diagnosis work,which integrates feature fusion and RNN classification into a synthesized model.