基于改进的隐马尔科夫模型的网页新闻关键信息抽取

Extraction of Key Information in Web News Based on Improved Hidden Markov Model

【目的】通过隐马尔科夫模型解决新闻网页中标题、日期、来源、正文等关键信息抽取问题,并根据应用场景对算法做出改进以提高抽取效果。【方法】将网页文档转为DOM树并进行预处理,映射待抽取信息项为状态,映射待抽取观测项为词汇,研究隐马尔科夫模型在网页新闻关键信息抽取中的应用并对算法提出改进。【结果】使用隐马尔科夫模型的改进算法,在已构建抽取模型的网站中,平均准确率可达97%。【局限】抽取模型在分类能力上稍有不足,无法对细微差别信息进行准确抽取。【结论】该方法具有识别准确率高、建模能力强、训练数据小、训练速度快的优点。

[Objective]This paper aims to solve key information extraction problems in news web pages,such as title,date,source,and text,by Hidden Markov Model(HMM).[Methods]The web document was transformed into a DOM tree and preprocessed.The information items to be extracted were mapped to state,and the observation value of the extracted items was mapped to vocabulary.The application of HMM in key information extraction of web news was studied,and the algorithm was improved.[Results]Using the improved HMM algorithm,the accuracy rate can reach 97%on average in the websites.[Limitations]The extraction model is slightly insufficient in classification ability,and it is impossible to accurately extract the slightly differences.[Conclusions]The experiment proves that this method has the advantages of high recognition accuracy,strong modeling ability,and fast training speed with small set of tracing data.