基于Transformer解码的端到端场景文本检测与识别算法

End-to-end scene text detection and recognition algorithm based on Transformer decoders

针对任意形状的场景文本检测与识别,提出一种新的端到端场景文本检测与识别算法。首先,引入了文本感知模块基于分割思想的检测分支从卷积网络提取的视觉特征中完成场景文本的检测;然后,由基于Transformer视觉模块和Transformer语言模块组成的识别分支对检测结果进行文本特征的编码;最后,由识别分支中的融合门融合编码的文本特征,输出场景文本。在Total-Text、ICDAR2013和ICDAR2015基准数据集上进行的实验结果表明,所提算法在召回率、准确率和F值上均表现出了优秀的性能,且时间效率具有一定的优势。

Aiming at the detection and recognition task of arbitrary shape text in scene,a novelty scene text detection and recognition algorithm which could be trained by end-to-end algorithm was proposed.Firstly,the detection branch of text aware module based on segmentation idea was introduced to detect scene text from visual features extracted by convolutional network.Then,a recognition branch based on Transformer vision module and Transformer language module encoded the text features of the detection results.Finally,the text features encoded by the fusion gate in the recognition branch were fused to output the scene text.The experimental results on the three benchmark datasets of Total-Text,ICDAR2013 and ICDAR2015 show that the proposed algorithm has excellent performance in recall,precision,F-score,and has certain advantages in efficiency.