电网OA系统非结构化文档内容自动化识别技术

Automatic identification technology of unstructured document content in power grid OA system

针对电网OA系统内存在大量的非结构化文档难以识别的问题,研究电网OA系统非结构化文档内容自动化识别技术。采用非直接转换方法,将非结构化数据先转换为采用XML文件承载的不完全结构化数据,利用SAX解析工具解析不完全结构化数据,采用Simhash算法对文本信息进行去重处理;采用TextRank算法提取文本内的关键词,根据关键词识别电网OA系统非结构化文档内容。测试结果显示,在海明距离与相似度阈值分别为10和70的条件下可获取较好的去重效果,关键词提取效果较好,具有推广价值。

To solve the problem that there are a large number of unstructured documents in power grid OA system,which are difficult to identify,the automatic identification technology of unstructured document content in power grid OA system is studied.The indirect conversion method is used to convert the unstructured data into the incomplete structured data carried by XML file,and the incomplete structured data is parsed by SAX parsing tool.In addition,the text information is de-duped by Simhash algorithm.The TextRank algorithm is used to extract the keywords in the text,and identify the unstructured document content of power grid OA system according to the keywords.The test results show that under the condition that the Hamming distance and similarity threshold are 10 and 70 respectively,good de-duplication effect can be obtained,and the keyword extraction effect is good,which has popularization value.