结合注意力机制和编码器—解码器架构的化学结构识别方法

Chemical structure recognition method based on attention mechanism and encoder-decoder architecture

目的 化学结构识别是化学和计算机视觉领域的一个重要问题,传统光学化学结构识别技术在复杂化学结构识别任务中易发生信息丢失或误识别的现象,同时又因为化学物质的结构多样性常导致其无法解析,识别效果不佳。而基于深度学习的模型通常具有网络结构复杂度高、上下文信息易丢失和识别率低的问题。为此,提出一种结合注意力机制和编码器—解码器架构的化学结构识别方法。方法 首先,使用改进的ResNet50(residual network)作为特征提取器抓取表征信息;其次,使用BLSTM(bi-directional long-short term memory)作为行编码器为ResNet50提取的表征信息加强空间信息;最后,使用去填充模块和基于覆盖注意力机制的LSTM(long short-term memory)网络作为模型解码器,对化学结构图像进行解码,将编码结果解码为SMILES(simplified molecular input line entry system)序列。结果 在Indigo、ChemDraw、CLEF(Conference and Labs of the Evaluation Forum)、JPO(Japanese Patent Office)、UOB(University of Birmingham)、USPTO(United States Patent and Trademark Office)、Staker、ACS(American Chemistry Society)、CASIA-CSDB(Institute of Automation of Chinese Academy of Sciences—Chemical Structure Database)和Mini CASIA-CSDB数据集上,所提方法识别准确率分别为71.1%、70.21%、45.8%、30.3%、53.02%、58.21%、43.39%、46.3%、84.42%和85.78%,高于SwimOCSR、Image2Mol和ChemPix模型得分。结论 与其他模型相比,本文方法通过少量训练集能够获得较高的识别准确率。

Objective Emerging digital and intelligent technologies have ushered in a new era of text recognition and interpretation.These advancements have greatly facilitated the ability to recognize and comprehend textual content originating from a variety of sources,including paper documents,photographs,and diverse contexts.One particularly noteworthy application of these technologies is in the field of chemical structure image recognition,where portable devices such as mobile phones and tablet PCs have become indispensable tools,playing a vital role in converting hand-drawn chemical structure images into machine-readable formats.They translate these intricate structures into human-readable representations,simultaneously highlighting relevant physical properties,chemical characteristics,and elemental compositions.These innovative models for chemical structure recognition serve as a bridge between hand-drawn representations and machine-interpretable data.This capability has made it feasible to electronically document complex scenarios,such as those encountered in classrooms and academic meetings.Notably,ongoing research has focused on developing encoderdecoder-based methods for mathematical expression recognition,which have shown promising results.However,the pivotal role of the quality and quantity of training data in shaping the performance of deep neural networks needs to be acknowledged.The current challenge lies in the absence of a comprehensive,high-quality dataset that is specifically tailored for chemical structure image recognition.This data deficiency poses a significant hurdle,impacting the optimization,generalization,and robustness of the models.Furthermore,the computational demands of real-time offline recognition on mobile devices remain a practical limitation.Method To address the aforementioned issues,we developed a chemical structure recognition model based on an encoder-decoder architecture.This model is capable of generating corresponding character representations,such as SMILES,from given chemical structure images.In the context of image-related tasks,the effectiveness of the encoder in extracting features from images and the decoder's ability to decode feature sequences directly impact the performance of the recognition task.The encoder is designed to efficiently model the input images,while the decoder should be able to comprehensively extract various features from the images,obtain accurate feature distributions,and encode them to establish feature maps.Therefore,we designed a feature extraction network based on ResNet-50 in the encoder,which adequately captures the two-dimensional structural information of chemical structure images.Furthermore,to enhance the effectiveness of information in feature maps,we introduced a row encoder based on bi-directional long-short term memory(BLSTM),reinforcing the spatial feature distribution weight through row encoding of feature maps.The decoder should be capable of accurately decoding the sequence information from the encoder's output.To align input sequence information with output characters and improve the model's memory and decoding capabilities for long sequences,we incorporated a coverage-attention mechanism into the decoder.Ultimately,the model can generate corresponding representations from input chemical structure images.Result For an objective evaluation of the performance of our model in this study,we conducted training on the Image2Mol and ChemPix models using the CASIA-CSDB(Institute of Automation,Chinese Academy of Sciences Chemical Structure Database) dataset.Subsequently,we performed performance testing on a range of datasets,including Indigo,ChemDraw,Conference and Labs of the Evaluation Forum(CLEF),Japanese Patent Office(JPO),University of Birmingham(UOB),United States Patent and Trademark Office(USPTO),Stacker,American Chemistry Society(ACS),CASIA-CSDB,and Mini CASIA-CSDB.Results demonstrated that our model achieved higher recognition accuracy when trained on small datasets and exhibited robust generalization capabilities.Furthermore,we compared our model with untrainable models such as SwimOCSR,MSE-DUDL,ChemGrapher,Image2Graph,and MolScribe.The comparison revealed that our model also exhibited commendable performance when compared with models trained on millions of images.Conclusion A chemical structure recognition method is introduced based on an encoder-decoder architecture.The method allows for the generation of SMILES strings from given chemical structure images.Experimental results demonstrate that the model achieves higher recognition accuracy when trained on small datasets and exhibits strong generalization capabilities.