基于多边形特征池化与融合的复杂文本检测

Complex text region detection based on polygon feature pooling and the transformer

文本检测在图像理解中发挥着重要的作用。基于深度学习的文本检测是当前的主流算法,包括单阶段方法和双阶段方法两类,而且后者的检测精度往往高于前者。双阶段的检测方法通常包含感兴趣区域特征池化操作,为进一步的检测和识别任务提供特定维度的局部区域特征。然而对于弯曲文本等复杂文本区域来说,现有的基于矩形感兴趣区域的池化方法不再适用,而基于点特征替代区域特征的方法又损失了空间信息。针对该问题,提出了一种基于多边形特征池化和Transformer的复杂文本区域检测方法。首先,将复杂文本区域检测中感兴趣区域进行多边形特征池化,将池化操作的区域形状从矩形拓展到多边形并且不需要借助其他形状进行拟合,即可将多边形区域对应的特征池化为固定维度的特征序列,避免了拟合过程中出现误差。进而,将池化后的特征视为具有空间关系的序列,然后利用Transformer融合视觉特征之间的上下文关系,降低训练难度,提升检测精确度。在包含弯曲文本等复杂文本情况的ICDAR2015、MLT、Total Text和CTW1500数据集上的测试实验结果表明,提出的双阶段检测算法能更好地提取感兴趣区域特征,并取得了比现有方法更好的检测结果。

Text detection plays an important role in image understanding,and deep-learning-based algorithms are popular methods including single-stage and two-stage methods.Usually,two-stage based text detection methods have a higher accuracy than the single stage based methods.The two-stage text detection method usually contains the feature pooling operation in the region of interests(RoI),which provides the local region features with fixed dimensions for further detection and recognition tasks.However,for complex text areas such as a curved text,the existing pooling methods based on the rectangular RoI are no longer applicable.Using point features instead of area features to solve the problem loses spatial information compared with area features.To address this issue,we propose a complex text region detection method based on polygon feature pooling and Transformer.First,we extend the feature pooling shape of RoI from the rectangle to the polygon,which does not need any shape fitting.and the features of polygon RoI with fixed dimensions are pooled,which avoids the error in the fitting process.Furthermore,the pooled polygon region features are regarded as context-sensitive sequences,which are input to the Transformer to fuse the context of the visual feature to reduce the training difficulties and improves the detection accuracy.Our experiments on the complex text region datasets,such as ICDAR2015,MLT,Total Text and CTW1500,show that the proposed two-stage detection algorithm can extract the features of RoI very well and achieves better detection results than the state-of-the-art methods.